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PREFACE 


The central thought in writing this book has been the adjust- 
ment of data, with emphasis on scattered portibns of the topic that 
are difficult to find elsewhere, and which in my opinion are destined 
to assume increased importance in the future. Some of the topics 
that in the past have been thought to be important in statistics 
and least squares are conspicuously absent here, or receive only 
scant mention. It must be confessed that this circumstance 
arises partly by choice. 

The intention has been to produce a book for reference, and 
also for a terct. Some differential calculus is used in the develop- 
ment of the general theory in Chapter IV, but it is not necessary 
to be able to foUow this development in order to apply the recom- 
mended procedures, or to interpret the results of the calculations. 
The main prerequisite is knowledge and experience in the subject 
matter. 

The reader must not expect to find in this book an account of 
statistical methods for all occasions. It supplements : it does not 
supplant. There has not been in my mmd any hope of covering 
the entire field, of least squares. For instance, recent contribu- 
tions from Hotelling, Wald, and Churchill Eisenhart have regret- 
fully been omitted. Au attempt to include them would have 
meant an unpredictable delay in publication. 

Possibly the reader will see here the interpretation of adjusted 
values in a new light, owing to my appreciation of the powerful 
stimulus of Shewhart’s contributions to statistical procedures and 
the philosophy of science. The student is first introduced to some 
basic statistical concepts, and in particular he is asked to view a 
method of adjustment as a way of arriving at a figure tfiaf can be 
used for a given purpose — in other words, for action. An abun- 
dance of procedures and skeleton table forms for numerical calcula- 
tion is provided for immediate adaptation to many kinds of prob- 
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leinfl'met in practice. It can be said that all of the recommended 
procedures have been tested in use, many of th^ in mass pro- 
duction. For the first time, a method for adjusting the obsei-va- 
tions (finding the calculated points corresponding to the observed 
points) is provided for the circumstance in which both the x and 
y coordinates are subject to error. The insidious phenomenon 
of the instability of equations is introduced, even though inade- 
quately, and the reader can at least claim acquaintance with it. 

The successful introduction of sampling into the 1940 Census of 
Population, aside from being a manifestation of wisdom and fore- 
sight on the part of Dr. Philip M. Hauser, Assistant Director of 
the Census, and Dr. Leon E. Truesdell, Chief Statistician for 
Population, brought with it a host of unsolved statistical problems. 
One of these was the adjustment of sample frequencies to known 
Tnnrgmfll totals, solutions to which are given in Chapter VII. 
With the subsequent rapid growth of sampling in the conduct of 
many social and economic survey of local and national scope, the 
mclusion of such methods may turn out to be timely. 

Different lands of problems of adjustment (e.g., geodesy on the 
one hand and curve fitting on the other) are here unified and 
brought under one general principle and one solution. The dis- 
tinctions between different kinds of problems are left where they 
belong, namely, in the conditions that the adjusted values are 
subjected to (Ch. IV). Unfortunately and inadvertently, intellec- 
tual gulfs have grown up between writers in statistics, least squares, 
and curve fitting. Each of the three groups has gone its own 
way, rediscovering developments long since discovered by the 
others, or — what is worse — not rediscovering them. Here the 
reader will find contributions from all three groups, and he will 
perceive that they are complementary. 

The methods of this book were developed over a period of 
sixteen years in the government service, during which I have had 
the pleasure of assisting colleagues in many branches of science. 
The manuscript originated in notes kept during my statistical 
practice, and to meet the need of text material for classes taught 
m the Graduate School of the Department of Agriculture. A 
mimeographed edition of portions of this book appeared in 1938 



PREFACE 


V 


under the title Least Squares as a publication of the Graduate 
School. Many of the calculations and procedures were worked 
out by my wife, Lola S. Doming. A number of helpful comments 
came from Professor W. G. Cochran, who kindly read the galley 
proof. Extensive contributions in the text, and help in reading 
proof, have come from several of my colleagues and assistants in 
the Census, notably Mr. Samuel W. Greenhouse, now with the 
armed forces, and Mr. Jacob E. Liebermau. 


Wabhinotoh 
Augvst 1943 


W.E.D. 
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Part A 

SOME SIMPLE ADJUSTMENTS 

CHAPTER I 

ON THE MEANING OE ADJUSTMENT 

1. Some remarks on the problem of adjustment. Before learn- 
ing how to use least squares, or any other method of adjustment, 
one might rightfully ask what is accomplished by procedures of 
adjustment, and what is the purpose of using them? 

In the first place it must be recognized that any measurement is 
the result of doing something — applying some operation. Some 
procedure is carried out, and some number is written down as a 
result. In the second place it must be understood that the purpose 
of taking the measurement is to use it for doing something, The 
object of taking data is to provide a basis for action. 

If you were to measure a table with the idea of ordering a plate 
glass top for it, you would use a rule, tape, or yardstick, and 
measure it. The procedure of laying down the rule, counting the 
number of feet, estimating the number of inches and fractions of 
the last foot, and recording the figure, constitutes the operation of 
measurement. The action, in this case, consists of ordering a plate 
glass of a certain size. The measurement provides a basis for the 
action. If the measurement is wrong by so great an amount that 
the glass is unfit for the purpose intended when it arrives, then the 
figure has led us to the wrong action. 

You might repeat the operation of measurement, especially if 
the length is required to the nearest sixteenth of an inch. What- 
ever the exactness required, the problem is fundamentally the 
same. One takes a measurement — that is, one carries out an 
operation — and thus gets a certain result (a number), and writes 
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SOME SIMPLE ADJUSTMENTS 
it down. WhyBhould he repeat the operation? The answer may 

be oontoed in one or both of two statements: (a) to get a better 

value for the purpose intended, by adjusting the observations; 
( 6 ) to gain some assurance that he is following the procedure 
intended. The latter is often more important, though also more 
difficult. Methods of adjustment assist us in both questions. 

As has been said above, the object of talcing data is to provide 
a basis for action, and an adjusted value is a derived number that can 
be used for the purpose vnlended, if it is possible to be had from the 
data presented for adjustment. 

The principle of least squares provides a method for getting an 
adjusted value. It can be applied whether or not the data are 
worth adjusting, but the results are useful only when the data are 
good in the first place; no purely mathematical procedure can 
TnB. 1 rp. a good figure out of any number of bad ones. Data not in 
statistical control — i.e., not random, are not usefully adjusted. 
It is important to know when data are worth adjusting. A partial 
answer will be arrived at in this section. 

Suppose that one were to repeat the operation of measurement 
n times, thus getting n numbers for the length of the table, denoted 
08 »i, ®2i !® 8 i • • * I ^n’ The problem is to adjust these observations, 

Observation number Observed value 

1 Zl 

2 at 

3 xs 


n x„ 


i.e., to derive from them a number that can be used as the length of 
the table, for ordering the glass. Assuming that the procedure is 
being followed correctly, one must answer the questiop : would the 
mean of the n measurements be better than any one of the measure- 
ments drawn at random? , Would the median of these n observed 
values be better than the mean? Would it be still better to 
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average the greatest and least of the n observed values? Why not 
just take any one of the observed values and use it? We sHhU 
proceed to some considerations that may help to provide useful 
answers to those questions. 

A statistician is expected to make better adjustments than any- 
one else. That is his business. However, the statistician must 
insist that he be rated, not on some individual adjustment, but in a 
population of adjustments, that is, on a “ long run ” of adjustments. 
If, in the long run he has a greater percentage of satisfactory results 
than anyone else could have gotten, then his method of adjustment 
is better. In isolated instances, his results may not be so good as 
those obtained by someone else, yet his method may be better in 
the sense just stated. 

Any measurement or any adjusted value is a prediction in the 
sense that the number that we are going to use for the length of a 
table is about what we should expect anyone else to get if he were 
to measure the table. As a matter of fact, every empirical scien- 
tific statement is a prediction, because, no matter how many times 
it has been confirmed in the past, it is always subject to future con- 
firmation by experiment. Any measurement is but one term in a 
sequence of terms (results) that actually or theoretically might yet 
be taken by repeated applications of the operation of measurement. 
It is important to realize that it is not the one measurement, alone, 
but its relation to the rest of the sequence that is of interest. We 
should not risk designating a measurement a measurement if we 
did not think that it could be duplicated within stated limits by 
future measurements, and that future action would bear out the 
usefulness of the number so designated. 

2. Randomness and the importance of order. In attempting to 
answer tlie questions that have been raised in the preceding para- 
graphs, let us make a chart, showing the results of repeating our 
operation of measurement. Let us plot the observed values as 
ordinates and the observation numbers as the abscissas. Suppose 
the chart has the appearance of Pig. 1. The observations show a 
trend. Under such circumstances should we take the average, 
median, or any other function of these observations for an adjusted 
value? The answer is m. Something is wrong with the procedure 
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or the meaeurmg mstrument. The first thing to do is to find out 

^^it^ toat the trend is recognizable only when there are 

anumber of measurements. If only one or two measurements had 
been taken, the trend and the existence of any difficulty would not 

have been recognized, and the glass ordered would not fit. 

Now let us do something else. Suppose that each number in the 
above table is written on a poker chip. Let these chips be physi- 
cally similar, put into a bowl and thoroughly stirred, and then 
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ORDER OF OBSERVATION 

Fig. 1. A chart showing the observed value plcrtted against the observation 
' number. The observations exhibit a trend. 


drawn one at a time with shuffling between draws. With care, 
this operation of. getting numbers will be a random operation, and 
will accordingly produce a random sequence. One actual random 
operation gave the sequence of numbers shown in Fig. 2. 

Just what is the difierence between Figs. 1 and 27 The numbers 
plotted are the same, i.e., for every number diown in one chart, 
there is the same number in the other. What is different, then? 
The order of the observations is different. The original order has 
been destroyed by drawing the numbers from a bowl, and there is 
no more trend. On this account alone, the adions that would be 
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taken on the basis of the two charts are entirely different. The 
action based on the series of n m^urements shown in Fig. 1 would 
be first to try to find out what is the trouble with the procedure of 
measurement that produced the n observations. Ordering the 
glass would come second; we should defer ordering it until we get 
better (more useful) measurements. On the other- hand, if Kg. 2 
had been the result of the actual measurements, we could go ahead 
and order the glass at once. It is not alone the observed values 
that count; their relation to one another in the order of production is 
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Fia. 2. The observations here exhibit randomness. These are the same 
numbers as shown in Fig. 1, but their order has been made random by draw- 
, ing them from a bowl. 


also important. 

When the trend of Fig. 1 occurs in actual observation with a 
measuring instrument, we immediately suspect something is wrong, 
and we try to find the difficulty. But if a trend like this were to 
occur under the ideal conditions of sampling (drawing the numbers 
from a bowl), we have no suspicions, but simply accept this result 
as one of the things that is going to happen once in a while. 

Let it be noted that trends and other patterns resulting from 
repeated observations occur not only in physical measurements, 
but also in the social sciences. For instance, in a survey that is 
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repeated monthly, if a person repeate^y answers the same ques- 
tionnaire, his answer may vary from tune to time, and may even 
show a trend, even though his status in life, measured by usual 
standards, has not changed appreciably. Repetition alone may 
be the cause of more careful attention to the details of the ques- 
tions, gradually bringing about a different evaluation of the 
gpTtiB circumstanoes, resulting in a trend. Moreover, repetition 
of a question month by month may actually produce a trend; 
for instance, if a housewife weighs out her flour week by week in 
order to record for some survey the amount she uses, she may 
become flour conscious and gradually use more or less than she 
did before. 

Results like the points in Fig. 2 show stability, or randomness, 
and can be statistically adjusted to get a figure that can be used. 
It is to be noted that a rather large number of measurements is 
required before one can say that the operation of measurement is 
random. Visual inspection of the chart is often sufficient, but 
the more dependable Shewhart criterion of randomness^ can be 
used if desired. The main thing is to have enough observations, 
and to plot them. With enough experience in using a particular 
method of measurement it may not be necessary to do this. The 
point is that before the observations can be adjusted, they must 
arise from a random operation. 

The adjustment itself may be a very simple procedure. It 
might consist of merely picking out any one of the n observations 
in Fig. 2 by lot, as one might be willing to do if (after randomness 
is assured) the measurements are all seen to lie within a band that 
is narrower than the requirement. Even if one of the n observa^- 
tions is picked out by lot, the other observations of the sequence 
are not thrown away; they all provide information. Together 
they, perform two functions: (i) they help to demonstrate the 
randomness of the operation of measurement, and (ii) they show 

^The Shewhait criterion of randonmess is described in hie book entitled 
Econmic Cotdrol qf QuatUy of Manufactured Product (Van Noetmnd, 1931) • 
also in his book Staiistical Method from the Viewpoint of Quality Control (The 
Graduate S^ool, Department of Agriculture, 1939). It is described and used 
,n the pmpMet entitied " Control Chart Method of Controlling Quality during 
Induction (American Standards Association, 29 West 39th St., Now York, 
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that the band of variation is so small that any one of them alone 
will suffice. 

The method of adjustment might of course be slightly more 
complicated. One might take the mean, or the median, of the n 
observations. The mean is in fact the least squares adjustment, 
as we shall learn in Chapter II. One could also conceivably split 
the difference between the greatest and least of the observations 
to get an adjusted value. 

The advantage of these slightly more complicated methods of 
adjustments is that if they are carried out for repeated sets of n 
measurements, the adjusted values so produced will fall within a 
narrower band than the band corresponding to the original obser- 
vations. For most random operations, the least squares adjust- 
ments will show the narrowest band of all, and this is a very prac- 
tical argument in favor of least squares. 

3. Performing a simple adjustment. Simple problems are al- 
ways best for illustration: if we can understand simple problems, 
there is some hope that we can under- 
stand more complicated ones. One 
of the best to look at from the stand- 
point of adjustment is a plane tri- 
angle in which the three angles have 
been observed by some angular meas- 
uring instrument, such as a transit 
or a protractor. In the triangle of 
Fig. 3 the three angles have been 
measured once each, with the results 
shown in the table. The observed 
an^es add up to 179° 30'. Suppose 
we demand that the angles be ad- 
justed so that their sum is 180° ex- 
actly. Two methods of adjustment 
mi^t at times suggest themselves. 


I 



Fio. 3, The three angles of a 
triangle have been measured. 
The sumoftheadjuetedangles 
ia forced to be 180°. 


Method 1. Distribute the 30' deficiency amongst the three 
angles in proportion to size. 

Method 2. Distribute the 30' deficiency equally amongst the 
three angles. 
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Angle 

Observed 

Adjusted 

Method 1 

Method 2 

1 

120*07' 

120° 27' 

120° 17' 

2 

38 23 

38 20 

38 33 

3 

21 0 

21 04 

21 10 

Sum 

178° 30' 




The two methods of adjustment give the results shown in the 
table. Which do you prefer? Either is simple enough, and your 
preference will be easily settled, depending on the circumstances. 
If the protractor is correctly graduated, then the measurements 
of an angle may be randomly distributed about its true value, 
resembling in a fashion those in Fig. 2. Under such circumstances 
one would prefer Method 2. If on the other hand the protractor 
had been stretched in its manufacture so that the 180“ index 
actually extends through more than half a circumference, then, 
though the measurements of any angle be randomly distributed, 
th^ will be distributed around a value’ that is too small. Under 
such circumstances, angular measurements need to be corrected 
by small additions, proportionate to the size of the angle measured. 
This is what Method 1 calls for. We can not say that either of the 
two methods is better; each has its place, depending on the cir- 
cumstances. (Cf. Sec. 5 also.) . 

As we shall see later (Sec. 31), Method 2 is the least squares 
adjustment under the assumption that the protractor is correctly 
graduated. Thus, the method of least squares seems to lead to a 
simple and common sense procedure. It will be so wherever the 
problem is simple enough to visualize. In a later chapter we ahall 
return to the problem of the triangle, in which the least squares 
procedure will be worked out for more complicated situations, in 
which the angles have been measured more than once, or an unequal 
number of times, and the sides may have been measured also 
(Sec. 34; pp. 74 ff.). 

Another simple example is a line that has been divided into 
segments, and some action is to be based on their lengths. The 
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observed lengths of the segments do not add to the observed length 
of the whole line, and an adjustment is required. A very Himp la 
procedure would be to apportion the excess or deficiency equally 
amonpt the segments and the whole line: if the segments are in 
excess, their observed values will each be decreased by a certain 
amount, and the observed value of the whole line will be increased 
by the same amount, this amount being the excess divided by 
n + 1, where n is the number of segments. This adjustment is 
very easily applied. It aotually is the least squares adjustment. 

More complicated problems of this . 

type occur when the segments are * ® c o e 

not all measured the same number Fio. 4. ThelineAEandits 
of times, or are measured with instru- fourBegmentahavebeenmeafi- 
ments of different precisions. Such The Bumofthoadjuated 
problems will form the object of later 
attention, but we pause here to note 

that in this simple case the least squares procedure provides an 
easy and satisfactory adjustment. (See Exercise 2, p. 86.) 

In more complicated problems, it is not so easy to picture what 
happens in the adjustment, but we shall be able to apply the same 
principles in working them out. Problems of curve fitting are 
essentially the same nature as the geometric illustrations just 
given; in each problem the adjustment consists of altering the 
observed values in order to satisfy certain conditions that we 
decide to impose on the adjusted values. 

4. Least squares adjustments often easy. It is sometimes sup- 
posed that the method of least squares is more difficult than most 
methods to carry out. This is not always so ; least squares is often 
the simplest and most satisfying of all known methods. In many 
problems, normal equations are not required. It all depends on 
what conditions are to be imposed, or bow rigidly the user insists 
on fulfilling them. There are some problems in which least 
squares provides the only known method at any price, as for in- 
stance, complicated problems in triangulation and geodesy; also 
the adjustment of the observations in curve fitting when the 
weights vary and more particularly when both coordinates are 
subject to errors of observation. We have already seen some 
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examples in which the least squares adjustment is simple and 
direct; for instance, it was noted in connexion with the adjust- 
ment of the observations in Fig. 2 that the least squares adjust- 
ment happens to be identical with the mean of the n observations 
— and calculating a mean is usually a simple enough procedure. 
We saw likewise that in the adjustment of the triangle by Method 2, 
the least squares adjustment turned out to be merely the equal 
distribution of the deficiency amongst the three angles. The least 
squares adjustment of the segments of the line in the last section 
is moreover simple enough, again bemg merely an equal distribu- 
tion of the deficiency or excess of the segments. Students who 
have fitted polynomials in the form of orthogonal polynomials will 
realize that the method of least squares, though perhaps not'simple, 
is at least a routine matter, not involving the solution of normal 
equations. There is another illustration, contained in Chapter 
VII, in which tables of frequencies obtained by sample surveys 
are adjusted to expected marginal totals that are obtained from 
other considerations, such as a complete count; here again the 
adjustment can be made rapidly without the solution of normal 
equations. One could go on and point out many other problems 
in which the least squares adjustment is about as simple to carry 
out as any that could be devised, in view of the conditions imposed 
on the adjusted values. 

6. Statistical methods and correction for biases. There is 
another kind of adjustment, which might be referred to as an 
adjustment for bias. Laboratory instruments are often calibrated 
against a standard, and a correction factor applied to the measure- 
ments. Sunilar corrections are often required in canvasses in the 
social sciences. A mailed questionnaire, for example, usually 
requires corrections, because not everyone responds, and those 
that do not, form a class distinct from those that do. Moreover, 
the responses in a mailed questionnaire will be different from the 
responses in an interview questionnaire, even for questions worded 
identically. Increasing the size of the sample, or the number of 
observations, will decrease the sampling errors, but not the biases. 
A statistical adjustment is applied primarily to effect compromises 
with statistical fluctuations (sampling errors and errors of observa- 
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tion). A bias is never discovered or measured, nor has any mean- 
ing, unless two or more distinct methods of observation or experi- 
mentation are compared with each other. Statistical adjustments 
of data, together with the Shewhart statistical methods of quality 
control, are powerful tools in the detection of biases, difference 
in performance, deterioration or other changes in quality, the 
standardization of quality, and a host of important related 
problems. 

Simultaneous adjustment for bias and statistical huctuations 
can often be made, as when sample frequencies constituting obser- 
vations on the breakdown of a certain class of the population are 
adjusted to the known total of that class (Ch. VII), or when a line 
is forced to pass through the origin because theory and other 
related knowledge of the subject tell us that it should (Sec. 15, 
p. 31). In the triangle problem of Section 3, Method 1 simul- 
taneously corrects for a stretched or compressed scale, and for 
statistical fluctuations of the measurements; but we should not be 
in position to choose between Methods 1 and 2 without knowing 
somehow or other from other experience with it whether the pro- 
tractor scale i. is uniformly stretched or compressed, or ii. can be 
considered perfect. 

6. Repeated experimental results necessary for establishing a 
scientific law. It would be splendid if all action required in social, 
economic, and industrial planning could be based on scientific laws; 
but actually, so many of the laws remain yet to be discovered that 
most action must necessarily be taken on the basis of knowledge of 
the subject matter in related fields. Of course, it is true that 
action is often prompted by prejudices and whims, even when a 
scientific basis for action exists, but this is a failing of human 
nature, hardly a problem in mathematics or statistics. 

No one ejq)eriment by itself establishes a law, or a valid basis 
for action. It is the consistency of repeated results under a variety 
of conditions that establishes a law. The method of least squares 
can be applied to a single set of data, but no matter how carefully 
the least squares adjustment is carried out, the curve so fitted, or 
the observations so adjusted, do not have scientific validity unless 
there is other evidence at hand to show under what conditions the 
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same or similar results will be obtained, ^and how these conditions 

are to be brought about and controlled. 

A long series of experiments may provide the additional evidence 
that is Lded, particularly if the different experiments of the series 
are performed under a variety of conditions (diffeient tempeia- 
tures, climatic conditions, economic levels, etc.). If the data m 
each experiment are random or nearly so (see Fig. 2 and discussion), 
md if. 'the adjusted coordinates or the adjusted parameters in the 
fitted curve turn out to have about the same values, time^ 
time, without fail, a scientific law may be considered established, 
and the conditions under which it holds may be stated. 

Thus, to be more specific, it is not the standard error of a slope, 
as estimated from a single set of data, but rather the persistent 
oTv,niiT,«H of the standard error, or the persistent recurrence of the 
slope, in experiment after experiment, under a variety of condi- 
tions, that really attains scientific significance. By this we mean 
that use/ul predictions can be made regarding future elopes, and 
that we can say under what conditions these slopes will be main- 
tained. Eepeated patterns lie at the basis of scientific significance. 
Repeaied and repeatable good fits, and repeated and repeatable 
statistical significance, establish a scientific law. In science one 
is usually if not always studying the underlying system of forces 
(social, economic, mechanical, chemical, or whatever), in order to 
take action on the cause system, to regulate future product. 
Measurements or surveys already carried out on some one particu- 
lar batch of product (population of people, lot of industrial product, 
etc.), provide part but only part of the chain of evidence that is 
required for predictions with regard to data of the future. The 


’ "... it being justly esteemed an unpardonable temerity to judge the 
whole course of nature from one single experiment, however accurate or 
certain.” From. Hume’s An Enpmy Concerning Unman Understanding 
(London, 1748), section vii, part 2. 

“ But to argue, without analysis of the instances, from the mere fact that a 
given event has a frequency of 10 percent in the thousand instances imder 
observation, or even in a million instances, that ... it is likely to have a 
frequency near to 1/10 in a further set of observations, is . . . hardly an argu- 
ment at all.” J. M. Keynes, Treatise on Probability (Macmillan, 1929), 
p.407. 
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operationally verifiable meaning of a scientific law ia a prediction 
of future results, not a statement of past results. 

Every experiment in a series should be designed and performed 
with care and judgment, even if many more experiments are 
required. Likewise, the data of each experiment should be sum- 
marized in the most eSficient manner for comparison with the 
other sets of data. The importance of statistical theories and the 
design of experiments can not be overemphasized. Under condi- 
tions of randonmess, the method of least squares usually provides 
a good summary of an experiment by preserving most of the 
information in the data, provided the right curve is being fitted. 
In some problems the method of least squares is simple and easy 
to apply; in others it is difficult (Sec. 4). In some kinds of 
problems, no other method is known. What method of adjust- 
ment to use (least squares, free hand curves, etc.) is as much a 
matter of economics as science, and must be decided on the basis 
of time, costs, and results. It is more important to insist on having 
a series of experiments carried out under a variety of conditions, 
than to insist on using any particular method of adjustment, 

7. The nature of an adjustment A student of statistical theory 
may well wonder how the adjustment of data differs from other 
statistical calculations, and in particular the calculations that are 
performed in problems of estimation. A problem of adjustment 
might be identified as a problem in estimation in which the end 
product ia a set of adjusted values, which have been forced (ad- 
justed) to satisfy certain conditions. 

It is these conditions that distinguish one problem from another. 
In the triangle problem of Section 3, and later on in Sections 31 
and 34, the sum of the adjusted angles is forced tq be 180“. In 
adjusting the line and line segments of Section 3, the sum of the 
adjusted segments must equal the adjusted total length of the line. 
In curve fitting (Figs, 16 and 17, pp, 132 and 133) there are likewise 
conditions to be fulfilled, because the adjusted observations are 
forced to lie on a so-called calculated curve. The principle of 
least squares (Ch. II) remains the same in all these problems, but 
the different kinds of condition imposed on the adjusted values 
lead to different procedures in certain preliminary stages of the 
solution. 



CHAPTER II 

SIMPLE ttLUSTRATIONS OF CURVE FITTING 

8. The principle of least squares. Before going into the general 
problem of the adjustment of observa,tiona (Ch.^ IV), it will be 
helpful to apply least squares to some simple applications in curve 
fitting. Fortunately, simple problems afford nearly as much 
opportunity for thought in the field of statistical inference as the 
more complicated ones do. In all of them, simple or complicated, 
the ‘prind'ple of least sgmes requires the minimizing of the sum of 
the weighted squares of Oie residuals, This sum may be written as 

)S = E ID res® (1) 

The summation (denoted by £) of the weighted squares of the 
residuals is to be taken over all the observations that are subject 
to error. S is called the " sum of squares," or, more explicitly, the 
" sum of the weighted squares.” Weight will bo defined in Sec- 
tion 11. 

In curve fitting, either or both of the x and y observations may 
be subject to error. Accordingly, S will be written explicitly 
for the X residuals alone, or the y residuals alone, or both, depend- 
ing on the experimental conditions. For instance, later on, 
when both the x and y coordinates are in en'or we shall write 

5 = Z {wj/ + %F/) (2) 

Here 7, is an x residual, and F„ is a ^ residual (see Pig. 17 on p. 133). 
If only y is subject to error, the first term on the right is to be 
omitted, and if only x is subject to error, the second term is to be 
omitted. In this chapter we shall be content to deal with a few 
simple problems in which only one coordinate has error. 

The principle of least squares is the minimizing of 8, The 

14 
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metJiod of least squares is a rule or set of rules for proceeding with 
the actual computation. Here we shall try to learn both, and how 
to interpret the results. 

We may now define x® by the equation 

( 3 ) 

The symbol a denotes the standard error of observations of unit 
weight (Sec. 11). 

Now since o- is a constant in any one problem, x* is a TniniTmnm 
when /S is a minimum; hence we may thinlr of least squares not 
only as the minimizing of S, but also of x^- Least squares may 
also be considered the minimizing of the estimate a (ext), to be 
introduced in Section 13. Another way of looking at the problem 
is to say that the principle of least squares is the maximizing of 
P(x), and that we seek the solution that gives the greatest prob- 
ability on the chi-test. 

Remark. It is interesting to recall Gauss’ Theoria Motm 
statement of the principle of least squares, in particular his 
recognition of the occasional need for compounding errors of 
different dimensions (seconds of arc, seconds of time, length, 
weight, etc.). In curve fitting, this compounding is exempli- 
fied as explained above, namely, by taking account of the errors 
in both the x and y coordinates, when both are subject to error, 
just as one would take account of the errors in both the angles 
and the sides of a triangle (Sec. 34). The following quotation 
from Gauss is talcen from his Theoria Motus Corporwm Codeaiium 
(Hamburg, 1809), Art. 179. His h is weight, written in Eq. 2 
and elsewhere as w. His sum hhw -f h'h'v'v' + h"h"i/'v" + 

■ ■ ■ is the jS of Eq. 1. 

"... quamobrem systema maxhne probabile valorum pro 
quantitatibus p, g, r, s, etc., id erit, ubi aggregatum + 
h'h'v'v' + h"h"v"v" + etc., i.e., ubi summa guadraiontm dif- 
ferentiarum inter valores revera bbservatoa et comjnUatos per 
numeroa qui praedaionia gradum metiuntur mutUplicatarum fit 
minimum. Hoc pacto ne neoessarium quidem est, ut funo- 
tiones V, V', V", etc., ad quantitates homogeneas referantur. 

Bed heterogeneas quoque (e.g., minuta secunda arcuum et 
temporis) repraesentare poterunt, si modo rationem errorum, 
qui in singulis aeque facile committi potuerunt, aestimare licet.” 
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This was Gauss’ enunciation of the pmoiple of least squares 
in 1809 In 1823 in his Thema Combinaiionia Observationum 
BrrarOm Minimis Obnoxiae, he took the view that one seeks 
values for the adjusted observations and param^ers which 
render the variance of the parameters a minimum.^ Both points 
of view are arbitrary, and are justifiable only m experience. 
Fortunately, both points of view lead to the same identical least 
squares solution. An article by A. C. Aitken and H. Silver- 
stone “ On the estimation of statistical parameters {Proc. 

Royd Soc. Edivtmrgh, vol. bd, 1942; pp. 186-194), is instructive. 

9. “The simplest example of curve fitting — the single sample. 
Let a single sample of n controlled^ (random) observations be 


X 



Fig. 5. Ten observations 

an, as, • • • , aio 

of equal precision (equal weight) are made on an unknown magnitude a. 
The true points are connected by the simple relation x = a, hence x = a is 
the curve to be fitted. The least squares value of a turns out to be S, the 
mean of the ten observations, x = a is the “ true curve," and x = 3 is the 
" calculated curve.” The calculated points are shown by the crosses; they 
all lie on the caleulated curve. This is the simplest problem in curve fitting. 
Compare with Fig. 6, page 25, and with Fig. 16, page 132. 

made on some magnitude, such as the length of a table. Suppose 
that it is desired to derive an adjusted value a, from these observa- 

^For a partial explanation of controlled observationB and randomness, 
see the first chapter. 
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tloDS. W 0 look upon the problem as one in fitting the curve 

X = a (4) 

i 

to the n observations. This is the simplest of all curves; it is 
merely a horizontal line (Fig. 5). It contains but one adjustable 
constant or parameter, namely, a. This parameter a is now to be 
determined. The method of least squares will be illustrated. 

The problem is to minimize S, the sum of the weighted squares 
of the residuals. The observations are aU of equal weight, since 
by supposition they appear to have been drawn all from the same 
bowl. We shall therefore let all the weights be unity. If x 
denotes an observation, then x — o is the corresponding residual, 
gdnce, by definition. 

Residual = Observed — Calculated 

The square of the residual will be (x — a)®; hence the sum 

(5) 

will, be the quantity to be minimized. The sign 22 means that 
the squares of all the residuals are to be summed. 

Here the y coordinates of the points are merely the ordinal num- 
bers of the observations (Ist, 2d, 3d, etc.). The y coordinates are 
of course without error here, so only z residuals appear in the expres- 
sion for S. 

Now the n observations, having once been made, can not be 
changed. They are constants. The only variable in Eq. 5 is the 
adjustable parameter a. By giving various values to a, S is made 
to take on various values. There will be a minimum, and it will 
occur when the derivative 

dS „ ^ 

vanishes, that is to say, when 

(x — a) = 0 or S ® “ wo = 0 


( 6 ) 
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The least equaraB value of a is accordingly 

a = i2a:s® = the mean (7) 

n 

So the horizontal line a = 5 in %. 5 gives the smallest possible 
value to the sum of the squares of the residuals, and hence to 
The line a = » is the " calculated curve ; it is the “ curve ” 
X = a fitted by least squares. On this line lie all the “ calculated 
points,” these being the least squares estimates of the observed 
points. (In problem, the calculated points all have the same 
ordinate, namely, 25. Compare Pig. 5 with Pig. 17, p. 133.) 

The fit of this line may be judged by comparing the value of 
x2/(n - 1) with 0 - 2 . This is done by looking up P{x) for the 
observed value of x* corresponding to n — 1 degrees of freedom. 
This subject will be touched upon again in Section 13 and else- 
where. Tables for the use of the chi-test will be found in R. A. 
Pisher’s SMistkd Methods far Research Workers (Oliver and 
Boyd). 

Note that the value of <r is not required for the application of 
least squares, because whatever v is, x* is minimum when S 
is a minimum, a did not occur in Eq. 6. v is required, never- 
thdess, for the use of the chi-test for the fit of the curve. It is 
presumed to be obtained from previous experience with the 
measuring instrument. By the time enough data are gathered 
to attain and test for randomness, tr will be known closely 
enough. 

Note also that if s denotes the standard deviation of the n 
measurements, then x* = ns*/a\ and the minimized value of 
S is For a new sample of n observations there will bo a 
new mean, x, a new line, and a new x*. Any one value of x can 
form a basis for action only if there is evidence that future 
values of x would be closely the same. This evidence must 
come from experience with the procedure of measurement. 

10. The same problem with unequal weights, (a) Direct solvr- 
tion. In the preceding section, all the observations had equal 
weight, 2 they were “ drawn from the same bowl ” (Ch. I). Sup- 
pose now that the n observations * 1 , ig* • ■ ■ » ®n have weights wi, 

* The meaning of weight will be learned in the next section. 
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ii) 2 j • ■ ■ j Wni perhaps not all ec^uaL The observations are now 
drawn from bowls having the same mean, but perhaps various 
standard deviations. The procedure is formally very similar to 
what it was before. We are now to make the sum of the weighted 
squares of the residuals a mimmnm , bo we write 

iS = £ Wj(a:< - o)® (8) 

the fth residual being, as before, Xi - a. Here, the weight Wi is 
introduced because the weights are not all unity. As before, S 
is to be a minimum with respect to o. The derivative 

dS _ 

^ 2 2 fl) 


when set equal to zero gives 

2 Wi{Xi - a) = 0 


or 

whence 


® 2 =. 2 WiXi 

Z WiXi 
a = -= 

2m'< 


X 


(9) 

( 10 ) 


where $ is now the weighted mean of the n observations. In the 
event that wi = W 2 = • • • = w„, this result reduces to the previous 
value of a in Eq. 7. In other words, the problem of the preceding 
section (equal weights) was a special case of this one. 

The minimized value of 5 is here 

^ = 2 WiiXi — 2)^ = Z — x^'^Wi (Seep. 151.) (11) 

(6) Tabular solution. In Section 61 we shall see a systematic 
procedure for the solution of normal equations and for calculating 
the “reciprocal matrix,” in which are found the variance and 
product variance® coefficients; also we shall see the minimiz ed 
value of S calculated right along with the solution of the normal 
equations. In simple problems like the one just considered, there 
is only one normal equation (Eq. 9) and it is of course very easily 


° Following Aitken, the term product variance is used here rather than 
coyariaiice. 
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, ^ Pd 10^ Nevertheless, it is interesting to see how the 
solved (see Eq. lUl. Section 61 applies here, 

routine process lia t^^ tabulation, and perform the 

MW. Th. sutacript < m the eunmmthme has 
been omitted for the sake of brevity. 


Row 




X 1®® 


3 

II 


_ (S 1®®)'* 

j (S«>®)^ 


An eUipais {• • •) in the tabular army denotea a space wherein a 
number would ordinarily be entered in numencal caloulafaon, but 
in which it is not worth while to show the entry m symbols. 

Row I is the main equation; it is equivalent to Eq. 9. Each 
letter across the heading of the tabulation is to bo multiplied by 
the coefficient standing below it in Row I. Row 2 contains the 
sum of the wei^ted squares of the x values, measured from zero. 
The C column is filled in as shown. Row 3 comes by multiplying 
Row I through by ~l,wx/j:w. Row II comes by adding 
Rows 2 and 3. In the " 1 " column of Row II is found tho quan- 

tity 2 

S = Zwx^- or E - sB* 22 w 


as already derived in Eq. 11. iS is here seen as the initial sum of 
weighted squares, Ews^ in Row 2, reduced by the amount 
(E™)*/L'y’ h) take account of the fact that the readuals are 
finally measured from the fitted line » = a, instead of from 0. 
It is interesting to percave that this minimized value of S has 
come forth wiffwut to intermediate step of computing each residual 
after adjustment, then squaring it, and then adding them all 
together. Pimi1fl.T short cuts, due to Gauss, will he found also in 
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the more complex problem, as -will be seen later. (See Secs. 29, 
59, and 61; also compare -with Sec. 156.) 

Bow I solved for o pves 

2 wx 
a = — — 


in agreement with Eq. 10. However, if we use the C column in 
place of the “ 1 ” column in solving for o, we get l/£ w. Inter- 
preted, this means that 


1 1 
Wa XiW 


( 12 ) 


This solution can be looked upon as the one and only term ia the 
reciprocal matrix (to be encountered later in extended form; 
Sec. 61). This one term is the variance coefficient of o, which 
interpreted means that 

(S.E.ofa)2 = - = ;^ (12') 

Wa L U> 


The standard error of a thus decreases as the weights of the indi- 
vidual observations increase. This equation will be understood 
better after the discussion on weights has been read (next section). 

11. A digression to define weights. By definition, the weight Wf 
of the function / is inversely proportional to the variance of /. 
That is to say, l/wj is the vaname coeffident of /. In symbols, 

2 2 


ff® is simply a proportionality factor, and is evidently ihe variance 
of a function of unit weight, E be arbitrarily doubled, and Wf 
also doubled, o-/® is unafiected in value. 

For example, let f be x, the mean of the n observations xi, 
® 2 j ■ • • ) which are random variates talcen from a universe of 
standard deviation c, hence each of unit weight. Then, since the 
variance of S is a^/n, substitution of * for / in Eq, 13 ^ves 


0^ 

wfe = -5 = ji or 




of 

n 


(14) 


n 



„ SOW! BIMHJ! ADJOaTMENTS [S.». Ill 

La 

. 1 , 0 + „ ifl the weight of 2, and l/?i its variance 
whence we see ^ ^ observations were each of weight 

coefficient. O^f since weights are 

w instead of unity i j^ripndine as they do on the arbitraiy 

obs«rvatioHB wtrfd be (P/u), 
“toiSeedK^ouMbeonenthaemueb. lu to ease, 
therefore, Eq. 13 ^ves ^ 


ws = -2 = ^^ 


or (r= 


(15) 


nil) 


^ prim^^ conception of a weight is that of a repented obser- 
vation In Fisher’s terminology, the mean 2 of n observations 
irains n times as much m/omoiion as a single observation, ^ 
Concerning two functions fi and /a, it can be said at once fiom 

= ( 16 ) 

which says that the weights of two functions are inversely pro- 
portional to thor variances. 

Exercise 1 If Vi denotes the residual at point i, Wi the weight 

rftoX™.to,4aax- -(!/»•) Showtkattomay 

be written 



which says that x® is the sum of the squares of the residuals, each 
residual (7i) being measured in units of the standard error o-/v«» 
of the corresponding observation (compare with Eq. 20, next sec- 
tion). In other words, x* is the sum of the squares of the stm^ 
ardized reddvds. x® is therefore independent of the units used in 


*E. B. Wilson and Ruth R. Pufier, " Least squares and laws of population 
growth," Ptoc. Aim. Acad. Arts and Set. (Boston), vol. 68, August 1933. 



[Ch. II] SIMPLE ILLUSTRATIONS OF CURVE PITTING 


23 


measurement; a change from feet to inchea or centimeters, or 
from pounds to ounces or grams, changes the residuaJs, but not 
the standardized residuals, nor 

Exercise 2. When both x and y observations are subject to 
error, one may ■wish to designate the summation explicitly as 

as has already been indicated in Section 8. Show that this may be 
written 



which again says that is the sum of the squares of all the resid- 
uals, each one being measured in units of the standard error of the 
corresponding observation on the x oi y coordinate. So is* 
as before, the sum of the squares of the standardized residvals. 
The remarks in the preceding exercise still hold. 

Exercise 3. S, or the aiun of the weighted squares of the resid- 
uals, like x^, is also invariant to changes in units (as from pounds 
to ounces, etc.). But S is dependent on the arbitrary choice of o-, 
whereas x® is not. One -wei^t in the whole set is arbitrary, and 
the other's are related to it through Eq. 13; fixing this one weight 
is equivalent to fixing a. S can bo doubled by doubling all the 
weights, but this has no effect on x* because o-® would also be 
doubled. The least squares solution for a (and other parameters, 
if any, as in more complicated problems) is independent of cr^] the 
parameter or parameters that minimize S for one set of weights 
will also minimize it if all the weights are doubled. 

For another inteipietation of S In curve fitting, see Exercise 3 of 
Section 68, pagel46, where S is seen to be equal to the sum of WFt!^. 
Other exercises dealing with weights occur at the end of Chapter HI. 

12. A more complicated problem — several samples, (a) AU 

observations have the same precision. Let us suppose that n obser- 
vations of equal weight (equal precision), and all on the some 
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unknown, as for example those of Section 9, are arbitrarily sub- 
divided into m samples of ni, nz, •• • ,nm observations. We shall 
say that 

Xi is the mean of nj single observations. 

Za “ “ “ " ^^2 “ “ ■ 


V « « “ M ** " 

Now if HiTigle observations have unit weight, then it will follow 
from Eqs. 14 or 16 that the weights of the m means are 

wi = nifWs = n2, “ • , Wm = ihn 
We may now consider the m sample means to be m observations of 
weights respectively wi, ■ ■ ■ , ?im, to which the results of Section 10 
apply. The value of a that minimizes is then 

2 wX JljZi JI2Z2 -f- • • • -)- n,nXm 


X = 




Wl + "2 H (- 


(17) 


which follows from Eq. 10. This value of X is the weighted mean of 
the m samples. Actually, it is also the mean of the entire group 
of Til + % + ■ ‘ • + Wm single observations, since they arc all of 
the same weight (unity). Our result implies that when the 
residuals (7,) are reckoned from this value of X, the sum S or 
2) wV^ is a min i m um. By Eq. 11, page 19, its value is 

S = E n<(Zi - Z)* = E W - nX^ 

A schematic representation of the observations, residuals, and 
errors, and their relationships to the weighted mean, is shown in 
Eig. 6. 

Exercise 1. Show that when the residuals (7,-) are measured 
from the value of X shown in Eq. 17, the weighted sum of the 
residuals is zero. That is, E «'7 = 0. 

It is not to be inferred thatE ^7 = 0 in all least squares adjust- 
ments; see Remark 4 on page 182. 

Exercise S, The value of X is independent of the mode of 
dividing up the n observations, that is, the subgroups of nu ^ 2 , 
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> ■ •, and Tim observations can be formed from the n observations 
in any manner whatever. 



Fiq. 6. Thiee eeries of observations on a magnitude ft. 

m observations have mean Xi and standard deviation si 
tta •' “ “ Zs “ " " ss 

m “ “ " Xi •• •• " a» 

X is the weighted mean of the three series. The errors and residuaJs in the 

individual means Xi, Xi, X 3 are denoted by Ei, Zj, IS} and 7i, Ts, 7s respec- 
tively. The error in the weighted mean X is denoted by E. As the figure 
happens to be drawn, B, Bi, E 3 , 7i, and 73 are positive, and Bt and 7a are 
negative, as the arrows indicate. This case of curve fitting is intermediate 
between the simplest problem shown on page 16 and more difficult ones de- 
scribed in Chapter VIII. 

Exercise S. On the contrary, the value of S does depend on how 
the n observations are subdivided, and similarly for x®. (Noie: 
X® is just fS divided by 

(b) The precisions 0/ the single observations differ from one 
sample to another. Suppose that 

Xi is the mean of n\ observations from a population of 
standard deviation <ri. Then the variance of will 
be 


Xm is the mean of Tim observations from a population of 
standard deviation am. Then the variance of Xm is 
On? Irim. 
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ai, ff 2 , need not all be equal. For the weights of Xi, Z 2 , 
etc., we may take 

_ for the weight of Xi 


Wl 


<rr <ri* 


m 


0“* Tljd I, „ 

^"2 “ 72 - _.2 

ffa ff3 
W 3 


0 ^ (I (( 


lfi« = 71 = - 2 

®)n ®»n 

rim 




(18) 


ff® is arbitrary, because the weights are purely relative. Again, 
as in Sections 9 and 10, the problem is to fit the curve 

a: = 0 (4) 


The answer is already contained in Eq. 10 on p. 19, which applied 
to the present problem gives 


a 


2 wX 

2 w 


n-iXi 712X2, 


®r 


®2 


«1 , ”2 , 

„ 2 2 
®1 ®2 


+ 


nmX-m 


+ 


rim 


= X 


(19) 


The quantity X just defined is the weighted mean of the m sam- 
ples. il^iduals (7») reckoned from it make S or 2 Wi(Xi — X)® 
a minimum. 


The problem of part (6) reduces to that of part (a) if ai => 
ff 2 = ■ ■ ■ = ffm- It is interesting to see that o does not appear 
in the fraction of Eq. 19, i,e., X is independent of a. If cr“ were 
doubled, all weights would be doubled, but X would be un- 
altered. Likewise x® in Eq. 20 would be unaltered. (See the 
exercises in See. 11.) 
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Exerdse 4- Show that when the residuals are measured from X 
as dehned in Eq. 19, 


EwV = 0 


as was true also in part (a) of this section. (See Exercise 1, p. 24.) 
Note that x® can be written 


x' = 2: 


(Discrepancy between Xi and 
Variance of X,- about true mean 


(20) 


See Exercise 1 of Section 11, page 22. 

13. The estimates of o-, internal and external. Because of the 
distribution® of x* when the actual sampling (the experimental 
work) is described by the mathematical model here assumed, 
namely, normally distributed observations, the mean value of 
x^ in the long run is equal to fc, the number of independent residuab 
or " degrees of freedom.”® In any one experiment, x^ niay be 
larger or smaller than the average. For the problems of parts (o) 
and (b), the number is m — 1 because there is one relation (Eqs. 17 
or 19) between the m residuals and X. The unbiased’' estimate of 
(T* made by external consistency is found by calculating what value 
of (T forces x* to take its mean value fc. In other words, the esti- 

® Karl PoaiBon, Phil. Mag., vol. 50, 1900: pp. 167-175, A paper dealing 
more specifically with curve fitting of the kind here considered will be found in 
the J. Amsr. Stat, Assoc,, vol. 29, 1934: pp, 372-382; see also Phil. Mag., 
vol. 19, 1936: pp. 389^2. 

° The correction for the number of unknowns evaluated (one in this case) 
and the equivalent of setting the mean value of x’ equal to S divided by the 
number of observed quantities diminished by the number of unknowns evalu- 
ated were set forth by Gauss in his Theoria Comtnnalionis Observationv.m 
Erroribus Minimis Obnoxiae, Pars posterior (Gottingen, 1823; vol. 4 of his 
Werke), Art. 38. This correction is sometimes credited to Bessel, but the 
reference just given, which was kindly furnished by Dr. G. J. Lidstone, places 
the originality with Gauss. 

7 Unbiased in the sense that its mean value is o-^. 

‘ The terms external and internal consistency were introduced by Birge 
(Phys. Eev., vol. 40, 1932: pp. 207-227). The comparison of the two estimates 
(Sec. 13) is an application of the " analysis of variance," the essential features 
of which have long been recognized by physical scientists; see, for example, 
A. de Forest Palmer, Theory of MeaauTemmis (McGraw-Hill, 1912) pp. 60-71. 
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[Siac. 13] 


mated a eatiefies Eq. 3, page 15, whence comes the estimate 

K 


prom this equation, for the problem of Section 12b, we get 

„ .X £^7® 1 ^rua^ ^,2 

<r{&ct) = — ■ — = r £ 2 " “ -*1 

' k m — I ac 




This estimate is made from the external consistency of the data, 
i.e., from the fit of the “ curve ” Z = o. What we do in maldng 
the estimate <r(ea:<) is to say arbitrarily that does equal k. 
This is equivalent to sasdng Ihat P(x) is about ^ — not exactly ^ 
because of the skewness of the x* distribution, which, however, 
gradually disappears with increasing k. 

If we are not positive that all m samples came from populations 
having coincident means, we should have as an alternate hypothesis 
that the m population means m, ihi ' • Mn» arc not all identical. 
Now if one or more of them really are not equal to the others, 
is raised, on the average, to some value higher than a®; 
consequently in examining the hypothesis that w = fiz = • • • = fim, 
we should be interested in knowing if a® (ext) is significantly gi’eater 
than (T®, or, what is the same thing, if x® is significantly higher 
than k. This can be ascertained by looldng up P(x) in tables of 
chi-square. Of course, ^ can not be computed or compared with 
k unless (t is known. Or, to use Fisher’s table of s, one would set 



(23) 


and look up P(g) with Fisher’s ni as m — 1, and with ng equal to 
infinity, since ir® is here assumed known. In regard to the inter- 
pretation of P(z), see the small type in the next section. Tables 
and examples in the use of P (x) and P (?) will be found in Fisher's 
StaMstieal MeOiods for Research Workers (Oliver and Boyd); also 
in several other texts. ’ 

Now the 

Wt. o{X = Wx=. 

/Tj* 


(24) 
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and the 

(Est’d S.E. of 


Wx 


1 

{m “ l)wx 


2u)73 


1 

(m - l)wx 


S 


(26) 


There is also the estimate of a made from the internal consistmcy^ 
of the data, i.e., from the consistency of the observations -within 
samples. This is^° 


c^fint) = + 

Jll + lla + • • • + Wm “ Wi 


whence the 


(Est’d S.E. of 


a^iini) 

Wx 


(26) 

(27) 


wherein Wx has the value given in Eq. 24. Si, sa, •■•,8m are the 
standard de-viations of the m samples. 

The estimate by internal consistency is possible only if there 
are points in which there is more than one observation. When 
there is but one observation at each point, the esthnate of <r by 
internal consistency is not a pcmbility. 

14. Comparison of the two estimates — analysis of variance. 
As was mentioned in the preceding section, the estimate o-(ea!t) is 
valid only if the m populations have coincident means; if any two 
of the means jui, (iz, ’ • are unequal, orient) is, on the average, 
raised above v*. But, in contrast, the estimate o-(mt) is unaffected 
by inequalities among the means of the populations; so long as a 
remains the constant standard deviation of all of them, the average 
value of o^ijnt) is still It follows that a statistical test of the 
hypothesis in = /i 2 = ■ ■ • = Mm is to examine the ratio of the two 

^ See the reference to Biige on page 27. 

See, for example, Eq. 67 in Deming and Bilge’s Staiiatiedl Thlory qf 
Bttots (The Graduate School, The Department of Agriculture, Waahington, 
1934, 1938), p. 168. 
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estimates. To do this, we may foUow Fisher and take 



and look in his tables to see if 2 is " significantly ” different from 0. 

(In doing this, we use m - 1 for Fisher’s ni, and ni + na H + 

7 i,„ - m for his WaO If t is found to be so large that it lies beyond 
the 1 percent limit, we say there is “ statistical evidence ” that 
the data are not homogeneous, or that not all the fi,- arc equal; 
in other words, that the curve 

= a (29) 


is not a good fit. 

Remark. Such a calculation of “ significance ” takes account 
only of the numerioal data of this one experiment. An estimate 
of (7 is not to be regarded as a number that can be used in place 
of cr unless the observations have demonstrated randomness 
(Ch. I), and not unless the number of degrees of freedom (the 
denominator in Eqs. 21 or 26) amounts to 16 or 20, and pref- 
erably more. A broad baokpound of experience is necessary 
before one can say whether his experiment is carried out by 
demonstrably random methods. Moreover, oven in the state 
of randomness, it must be borne in mind that unless the number 
of degrees of freedom is very large, a new experiment will give 
new values of both a{ex() and (r{int), also of P(x) und P(z). 
Ordinarily, there will be a series of experiments, and a cor- 
responding series of P values. It is the conmknffii of the P 
values of the series, under a wide variety of conditions, and not 
the smallness of any one P value by itself that determines a 
basis for aotion, particularly when we ore dealing with a cause 
system underlying a scientific law (Ch. I). In the absence of a 
large number of experiments, related knowledge of the subject 
and scientific judgment must be relied on to a great extent in 
framing a oouise of action. Statistical “ significance " by 
itself is not a rational basis for action. 

16. Another simple problem — the slope of a line that is known 
to pass through the origin, (a) The y coordinates subject to error; 
x/ree of error. The equation to be fitted to the points in Fig. 7 is 

y = bx (30) 



[Ch. II] SIMPLE ILLUSTEATIONS OF CURVE FITTING 


31 


Let yi denote the observed ordinate at the ith point; then yi — hxi 
is the residual at that point. It is a vertical, or y residual, because 
the error is all in j/, by assumption. If Wi denotes the weight of yi, 
then the sum 


iSf = Z Wiiyi - bXiY (31) 

is to be minimized. We differ- 
entiate this with respect to b 
and obtain 

dS 

-^= -^'ZwfiSiivi-hxi) (32) 

Set equal to zero, this gives 
6 Z w’®* = Z wxy (33) 
whence 

Z wxy 


b = 





(34) 


PiQ. 7. A line known to peas through 
the origin. The elope is to be esti- 
mated from the observed points. 


The subscripts are omitted for convenience, w means the weight 
of a y observation, as before. 


Note that here, neither Tl res norT*. w • res is neceesarily zero, but 
that w • X • res = Q. The student should demonstrate these 
statements. (Cf. Remark 4, p. 182, for an extension of this note.) 

Special cases, i. Suppose that the weight of y is inversely 
proportional to x; i.e., the square of the standard error of y is 
proportional to x. Then Eq. 34 gives 

b = ^ (35) 

Here, Z = 0, as the student should prove. 

The result obtained in Eq. 35 has application in many prob- 
lems in the poial sciences. Sample surveys of {e.g.) vacancy are 
often taken in a dty or metropolitan district, by picking out 
certain blocks, or segments of blooks, and noting at every 
dwelling unit therein (or sometimes at every kth dwelling unit) 
whether that dwelling unit is vacant or occupied. If m Block i, 
or Segment i, it is found that there are Xi dwelling units, of 
which yt ore vacant, then when the survey is completed, the 
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.rttoted T»».W »<• 
may be taken aa 

TM...1 nnmbei of vacant dwe llinR unite in the eamplg 
fSl&iber of dwelling unite m the sample 

sample is not too or metropolitan district is sufficient. 

to S ,m ortunAte oI «» 

teS Sai of Mngton. in tbeir sample surveys of rent 

ISCication of Eq. 35 is to the Imtchability of eggs: 
the mS egg? £ L more hatch y (except for random scattered 
SeS^rbut also the greater the error m y, m absolute 

is used in Example IV at the end of the book, for a 
sample inventory of canned goods. 

, ii. Suppose'that all the y weights are e^al; i.o., all y obser- 
vations Lve the same standard error. Then Eq. 34 gives 


h 


(30) 


This is perhaps a more usual case than the preceding one, par- 
ticularly in engineering, physios, and chemistry. 

iii Suppose that the weight of an observation on y is in^rsely 
proportional to e*. By putting to = !/»*, we find from Eq. 34 
thatj 


(37) 


The letter n here stands for the number of points. Each ob- 
served pomt gives an observed slope y/x, and the least squares 
estimate obtained from all the pointa is in this case simply the 
average of all n observed elopes. 

' Private oonununioation to the author. 
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The distinction between Eqs. 36, 36, and 37 should be noted 
carefully. In Bq. 35 a point has more influence on 6 if it is far 
outlying, this influence being closely proportional to the distance 
of the point from the origin. In Eq. 36 the influence of a point 
is further accentuated by its distance from the origin. In Eq. 37 
the advantage of distance is completely removed, the final result 
being merely the average slope of the n rays joining the origin 
with the observed points. 

(6) The tabular solution of b, and its weight, and the mm S. 
This will be similar to the tabulation in Section lOb (q.v.). We 
enter in Row I, from Eq. 33, the coefficient of b under 6, and the 
right-hand member under the “ 1." Enter the weighted sum of 
squares of y in Row 2. Fill in the C column with 1 and 0 as shown. 


Row 

h = 1 

c 


I 

2 wa® 2 

1 


2 

2 ^2/* 


How obtained (cf. Sec. lOb) 


(2 waj/)* 


(By multiplying Row I through 


2 1''®® 


by —2 iixcy/^ tos*) 

II 

9! (L wai/)’ 

£ W V' 2 

2^wx^ 

... 

(By adding Rows 2 and 3) 


An ellipsis (■ ■ ■ ) in the tabular array denotes a space wherein a 
number would ordinarily be entered in numerical calculation, but in 
which It is not worth while to show the entry in symbols. 

Row I solved with the “ 1 " column gives 6 as in Eq, 34. 

Row I solved with the G column gives b = l/£ wx^, which means 
that 

The weight of & = wj = 2 (38) 

Row II shows the minimized value of jS in the “ 1 ” column, 
which is to say that 

)S or 2 w(y -hxf = ' 


(39) 
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Thus S is calculated in the tabular solution without the. necessity 
2^^t solving for b and the individual residuals. The mitial suin 
n sauares 22 in Row 2, is seen to be reduced by the amount 
(£^)2/£ the residuals being finally measured from the 

fitted line, instead of the a ams. . 

Here the external estimate of a will be found by writing 




S 


(Cf. Sec. 13.) 


(40) 


whence the 

(Eht’d S.E. of b)„t® 


8 S 

(m — l)uii (m — 1) v)x^ 


(41) 


16. The t test for the slope. In order to apply the Student t test 
to see if there is statistical evidmce that the calculated value of 
b is “ significantly different ” from some theoretical value, say B, 
we should write 


|B- b 

Est’d S.E. of b 


(42) 


and make the Student t test, using Fisher’s n equal to our m - 1. 
The region of rejection in the { distribution is to be chosen with 
due regard to admissible alternative slopes, which may bo greater 
or less than B. In the denominator of Eq. 42 we may use the 
estimate made by external consistency, or that made by internal 
consistency (Sec. 13). If were used in place of tr(exi) in 
Eq. 41, then we should have 


(Est’d S.E. of b)iJ = 


a^{int) 

Wb 


(T^{int) 

22 


(43) 


This would replace the denominator of Eq. 42, and the number of 
degrees of freedom (Fisher’s n) would be the total number of 
observations diminished by the number m. 

With regard to the interpretation of statistical tests, see the 
remark at the end of Section 14, page 30. 
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17. The * coordinates subject to error, y free of error, w,- will 
now denote the weight of a:,-. In place of Eq. 31 we now have 

S = (44) 

since here the y residuals are zero, and S is made up by squaring 
the X readuals. By differentiation 

, 2^ / y\ 

db ( 45 ) 

Set equal to zero this gives 

62; = r wy^, or 6 = (46) 

£ wxy 

Note the distinction between Eqs. 34 and 46. w in Eq. 46 is the 
weight of X, not y. 

For another derivation of Eq. 46, see Hint 1 in Exercise 12, 
following. 


ExsnciSBs 

Exercise 1. If 6 in Eq. 46 be distinguished as prove that 
between Eqs. 34 and 46 there exists the relation 

^ _ £ viyty T, WxXy 
h' 22 2 Wxy^ 

Exercise Z. Find the curve y = hx, also Wi, from the following 
data. 


X 


Wy 

X 

0.62 

1 

2 

0.96 

1 

3 

1.60 

2 

6 

2.65 

1 


y alone is subject to error. Use the tabular arrangement on 
page 33. 
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Exerdse 8. Find the curve y = h'x, also w^f. 


Xobi 

y 

Wx 

1.05 

0.6 

1 

1.00 

1.0 

1 

3.08 

1.5 

2 

4.86 

2.8 

1 


X alone is subject to error. Again use the tabular arrangement, 



Part B 

THE LEAST SQUARES SOLUTION OF 
MORE COMPLICATED PROBLEMS 

CHAPTER HI 

THE PROPAGATION OF ERROR 

1 

18. Small errors in functions of one variable. If /(ic) is a func- 
tion of X, the linear term in Taylor's series can often be used to 
express vdth sufficient accuracy the effect on f{x) of a small error 
in a;. Thus, if Aa: is the error in x, and A/ the resulting error in 
f{x), A/ and A® may be closely enough related by the equation 

A/ = fix) Aa: (1) 

This is the equation for the propagatim of error in a function of a 
single variable. f{x) is the first derivative of f{x), evaluated at 
the point x, fix). In practice, the true value of a: is not known, 
but it is usually sufficient to evaluate/(a:) at a near-by point, such 
as a point whose coordinates are determined experimentally. 
fix) remains constant while Aa: and A/ vary. 

The above equation says that the error in fix) will be propor- 
tional to the error in x. The derivative fix) is the factor of 
proportionality. The equation is not exact, i.e., the error in / is 
not strictly proportional to the error in x, except when /(s) is a 
linear function of x. It is close enough for actual use provided 
the error A® is small enough, or when the higher derivatives of 
fix) are small enough. Fortunately, in practice much experimental 
work, and most of the functions used, satisfy these requirements. 

The relation between the error in fix), and the approximation 
afforded by Eq. 1, is shown in Fig. 8. 

37 
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In a function, such as 

/(a) =0+6® (2) 

the higher derivatives (®), etc., are zero absolutely, and 


Eq. 1 reduces to 


A/ = 6 A® 


(3) 


which is exact for any error in x, however large. The error in 
/(is) wiU now be exactly proportional to the error in x. 


Kit 



PiQ. 8. Showing the relation between the onora in x and /(x), and the ap- 
proximation contained in the equation 

A/ =/'(x) Ax 

Ax is the error in x, A/ the error in the function /(x), Tho approximation in 
Eq. 1 is made by uaing the tangent to the curve in place of the curve itself. 

19. Small errors in functions of several variables. Taylor’s 
series can be extended to obtain expressions for small errors in 
functions of several variables. Thus, if is a function of x, y, z, 
and if they are in error by the amounts A®, Ly, Az, then F will be 
in error by some amount AF, which can be expressed closely 
enough as 

AF = Fa Ax + Fy Ay + F* Az - (4) 

Iprovided the errors Ax, Ay, and Az are small enough, or when the 
liigher derivatives are small enough. Here F*, Fy, F^ denote the 
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F, 


SF 3F „ dP 

dx’ 


( 5 ) 


which in practice are to be evaluated at the point z, y, z, or as 
near this point as is experimentally possible. 

Eq. 4 is the formula for the propagation of error in three variables. 
It can be extended to more variables simply by adding more terms 
of the same kind. 


Bq. 4 is written only through the first powers of A®, Ay, and 
As, because the rest of the Taylor series (involving the squares 
and higher powers and cross-products of As, Ay, and As) will be 
negligible if As, Ay, and As are not too large, or if the higher 
derivatives are small enough. In practice, the possible errors 
- Aa, Ay, and Az are limited in magnitude, and Eq. 4 is usually a 
good enough approximation for ordinary situations. In the 
event that F is linear in x, y, and z (as in Exercise 2 at the end 
of the chapter), there are no terms at all except the linear terms 
(i.e., there are no neglected terms), and Az, Ay, and As may 
then be ever so large without invalidating Eqs. 4, 7, 8, and 9. 
(Compare with the explanation in the preceding seotion for a 
function of a single variable, particularly the text accompanying 
Eqs. 2 and 3.) 


20. The propagation of mean square error or variance.v /Eq. 4 
leads to a relation between the mean square errors or the variances 
of X, y, z, and F, and hence also a relation between their standard 
errors and their weights. If we square each side of Eq. 4 we get 

Af^ = Axf + (E„ Ayf -f- (/f, Axf 2F^Fy Am Ay 

-1- 2FxFg Ax Az + 2PyF, Ay Aa -1 (6) 

Now let Ax, Ay, and Az talre on all possible values^ within their 
allowable ranges of variation. The derivatives Px, Fy, Fa, being 

^ In practice it may safely be assumed that the ranges of variation in As, 
Ay, and Aa are not large, wherefore the constancy of Fa, Fy, F, is usually not 
a difficulty. It is moreover presumed that the standard errors ax, ay, anch»r, 
actually do exist, as they always do in practice. (There are theoretical and 
attainable distributions of errors, in which the standard deviation la infinite. 
An example is the Cauchy distribution y = l/ir(l + a®), the like of which 
must be excepted.) 
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whence by Eq. 4, we calculate the approximation 

^ = -16.6929 X 0.1 - 0.6466 X 0.05 + 1 X 0.2 

= -1.4016 (16) 

Thie is to be compared with the exact value of AF, computed in 
Eq. 13. Other functions, and other values of Ax, Ay, At, would 
pve different degrees of approximation. In the development of 
the general problem in least squares, we shall be compelled to 
accept the approximations afifoi’ded by Eqs. 4, 7, 8, and 9. For- 
tunately, for purposes of action, the results are usually close 
enough. 


EximciBES 

In the foUawing exercises, independence of the observations is 
assumed, as in Eq. 8. 

Exercise 1. (a) The mean square error of the sum or difference 
of two numbers having equal precisions is twice the mean square 
error of either alone (assumed independent). 

(6) The root mean square error (standard error) of a sum or 
difference of two numbers having equal precisions is \/2 times 
the standard error of either alone. 

(c) The root mean square error of the sum of n observations 
of equal standard error, c, is <r^/n. 

(d) A sarveying party chains a distance of L feet. Show that 
the standard error of the measurement is proportional to \/'L, 

Exercise S. (o) If u is a linear function of the independent 
variables x, y, and %, say 

u = ttx hy cz 

then the root mean square errors are related by the equation 

er„® = oW + + c^ir/ 

A special case is contained in Exercise la. 

(6) If F is a linear function of the n independent variates 
* 1 , ® 2 , • • •, iSn of the form 

P = a\xi + 02 a !2 + • • • -]- o„a:„ 
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and if the variates xi, Xz, •••,»» are distributed with variances 
fi^i • • •) ®'n^» then F is distributed with variance 

Vjr® = + ■ • ■ + On^ffn^ 



(a) If M = axys, then 


which interpreted says that the squares of the percentage or relaMve 
mean square errors are additive (or the squares of the coefficients 
of variations are additive). 

(6) The result of part (a) can be written 

ffy® = trx® + ffy* + 

where U = logj u, X = log* x, etc., k being any base whatever. 
Exercise 4. If « = axy/ve, then 



The squares of the percentage errors are again additive. 

Exercise B. liu = oaTi^z', then 

(^’-(Ty+(^y+(^y 

Here the percentage errors are increased by the factors a, j3, y. 
(Exercises 3 and 4 are special cases of this.) 

Exercise 6. For the conditions of Exercises 4 and 5 the relations 
between the weights are respectively 

J L + J_ + J_ + J_ 

U^Wu X^Wx Z^Wg 

and 

1 ^ .y2 

y^Wy z^v)t 
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Exercise 7. (o) If A = irr® (A the area and r the radius, d the 
diameter, of a cirde), then if an error Ar be committed in measur- 
ing r, or Ad in d, the corresponding error in the area is closely 

2A 

AA = 2rr Ar = — Ar 

T 

whence 

AA „ Ar Ad 
= 2 — = 

A T a 

An error of 1 percent in either the radius or the diameter thus 
means about 2 percent error in the area. Also 

= 2 - and A^Wa = 

A r 

(This is a spedal case of Exercise 5.) 

(h) The measurements on the sides of a rectangle, a and 6, are 
subject to the errors Aa and Ah. Show that these errors are re- 
lated to the area A by the equation 

^ ^ ^ (The percentage errors 

A a h are thus additive.) 

(c) The same equation is satisfied by the area of an ellipse, a 
and h being the axes or semi-axes. 

Exercise 8. (a) If y is in error by the amount 8y, then In y is in 
error by appronmatdy 8y/y, if Sy is not too big.* 

(b) If logarithms to the base 10 are used, the error in log y is 
approximatdy 0.434 liy/y. 

(c) In particular, let y change from 16 to 16, as in Fig. 9. Then 
calculate the increment in log y by the approximate formula just 
derived, and compare it with the exact value of the increment. In 
other words, carry out the calculations mentioned in the legend of 
Fig. 9. 

® The abbreviatloa In is used here for “ logarithme naturel,” os is common 
in Europe, and among chemists everywhere. The abbreviation log will be 
used for a logarithm to base 10. 
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(d) Let 7 = In I/; then 


and 


^ a _ ^ 


Wy = y^Wy 

lo* y I y 


S log y • 0.02803 


1.20412 + 16.0 


■S y» 1.0 


I.IV609 -f IS,0 tRUE VALUE 


Fio. 9, IHuatrating the relation bet-ween an error iO' y and on error in 
log y. Here y changes by unity, and log y changes by 0.02803. The ap- 
proximate relation a log y = 0,434 dy/y gives 0.02896, which is to be compared 
with the exact value 0.02803. Smaller changes (smaller values of Sy) show 
better agreement, but even for this rather large value of iy the approximate 
relation would be adequate for many purposes. (See Exercise Sc, p. 44.) 

(This result is important; see Exercise 18 in Ch. X, p. 201.) 

(e) If 7 = log y, then 

.2 


ay 


= ^0.4343 


and 


-^ = 0.434=*-^ 

Wy fWy 


Exercise 9. Let u = oe*®, then 


where 


ffu® = or cp® = 

(7 = In w 
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MORE COMPLICATED PROBLEMS 


[Ssc. 22] 


Exerdse 10. The period of a simple pendulum is r= 27r\/(L/p). 
Show that if the length L is too long by one-tenth of a. percent, the 
clock will lose about 44 seconds per day. 

Exerdse 11. (a) Prove that if 1? is a function of x, and x a func- 
tion of (, then 

Wx Wt 


where F* denotes dF/dx, and Ft denotes dF/dt. 
(b) Prove from Eq. 9 that 


Wx - 1 

/dtV 

It) 

\dx/ 


Var X = 

\dij 

(Var denotes variance.) 



(ffa denotes the standard error 

<rii = 

l*r‘ ' 

of » ; (Tt the standard error of t . ) 

Exercise IS. 

Prove that when the line 


y = h'x 


is fitted to points for which x is subject to error and y free of error 
(as in Sec. 17), it turns out that the weight of b' is 

w = ^ L ^ E wzy’‘ 

Hird 1: From Section 15a, wherein y was subject to error, and 
X free of error, we had ; 

= S WgX* (Eq. 38, p. 33) 

Now look at Kg. 7 on page 31. ’ Viewed from the back it will 
appear like Pig. 10, and the equation of the line will be 

® = ^/ 2/ (v now free of error) 
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THE PROPAGATION OF ERROR 
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From the equations just written for h and its weight, we may 
now interchange x and y in the formula for 5 and write 


Jii_ 2^ 

6'^ L 


(Cf. Eq. 46, p. 36.) 


till = £ 

S? 


whence by the result of the preceding exercise, part (6), 



= ^ L ^ Q.E.D. 


X 



Fig. 7 when viewed from the back. 


Hivi 8: (Due to my colleague Morris H. Hansen.) Write 
ji ^ : T,Wzy’‘ 


•WxTXm + tSxsE2j/2 d i- lll*n»nF» 


(Cf. Eq. 46, p. 36.) 


dxi £ WxiBm 


Now make use of Eq. 8 on page 40 and get 


Wt! 


(£ 


Obi^ h'^ £ Wiiyi* 6'® 


= TTg £ tu^aiy Q.E.D. 
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Ex&dse 13- (®) Recompute the approximation to AF in 
Section 22 by using the derivatives Fy, F^ evaluated at * + 
Ar = 1 q u -1- Av = 1.0 , 2 + Az = 10.2, instead of at the imtial 
values of x, y, «. {Anmsr: AF = -1.3184. Note that the 

result is very close to that shown by Eq. 16.) 

(5) Show that when the derivative in Eq. 1 is evaluated at 
j + Aa: instead of at a:, the result for A/ differs only in squares 
and bi^er powers of Ax from the value obtained for Af by eval- 
uating the derivative at a:. , j . . 

(c) Prove a similar statement for Eq. 4 when the derivatives are 
evaluated at x + Ax, i/ + Aj/. z+Aa (In practice we are 
obliged to evaluate the derivatives at the observed points, not the 
adjusted points. Fortunately the distinction in the results is 
usually negligible.) 

Exercise U. If is the ^th residual from the line in Sec. 15, 
page 30, that is, if 


Then, since 
it follows that 


Vi = Vi - hxi 
i = rffy/a-g 
Vi = Vi- ricyh^Xi 


Show that the variance of 7 is 

Var7 = aV)=<^i/®(l-»’®) 

ffini; Since a and y are correlated, use Eq. 7 and find that 

Var 7 = (Tj® -I- + 2(-j'o'j/fft)ro'*o’ff 

= (rv*(l - r*) 

See Exercise 35 on page 177, where E 7i® is given explicitly. 



CHAPTER IV 

THE GENERAL PROBLEM m LEAST SQUARES 

23. Outline of the problem.^ As a result of any experiment or 
sample survey there vdll be observations, and when the adjustment 
is completed, to each observed value there will be a corresponding 
adjusted value. It is useful to introduce the concept of a true 
value, which is merely the average value that would result from 
repeating the experiment a large number of times in a state of 
randomness. In curve fitting, one can visualize the relation be- 
tween the observed, adjusted, and true coordinates, and it may be 
helpful to the reader to turn forward at this time to Pigs. 16 and 17 
on pages 132 and 133. 

In formulating the general problem we shall deal with the quan- 
tities listed in the table below — 

Observed quantities: Jfi, Za, •••, Znl Yi, Yi, Yn 

Their adjusted (or 

calculated) velues: ici, xa, •••, x„; yt, •••, 

Their weights: tVgi, uija, •••, Wm', Wyi, ‘"i Wyn 

Their true values: {i, fa, •••, n tja, % 

The residuals (obs'd - 

calc'd): Vai, Vrs, ■■■, Ym] Yyi) Vjia, ■■■, Yyn 

In geometrical problems, and other problems not involving 
parameters, the observations need not be considered as coordi- 
nates of observed points. 

The assumption will be made here that there is no correlation 
between the errors in the observations. This assumption covers 
a wide class of problems, but does fail to cover some. 

^ The development from here on is an amplification of three papers that 
appeared in the PM. Mag. The references are vol. 11, 1931: pp. 148-168; 
vol. 17, 1934: pp. 804-829; vol. 19, 1936: pp. 389-402. 

49 
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Tlie residuals (7) aie defined by equations typified by 

(Res. = obs’d - calc’d) (1) 


Fri = Xi - x^^ 
7yi =Yi-- Vi) 


It is the readuals that are actually calculated first, and in actual 
to equrfioM »» reverse Once tto r^<^c 

^ found, i ndiunled quantttia am cntaiinted by .ubteactmg 
ainh teaUunl in tan from to corresponding observation, no- 

cording to Eqs. 6 ahead. , , , . 

24. The conditions. The principle of least squares requires 

that the sum of the weighted squares of the residuals, 

S = Sto ■ res® (2) 

be made a minimum with respect to the adjusted values 
q. ... a: 1/1 wa, • • •, 2/«- But this is not a simple problem in 
the and minimum of functions, for here the adjusted 

values are related to one another. For example, in the case of 
measurements on the three angles of a plane triangle, we required 
that xi + ica + X8 = 180“ (see p. 7). In curve fitting, the prol> 
lem is further complicated by the fact that the conditions on tho 
adjusted values (xi) involve the estimates a, h, c of the unlmown 
parameters a, /3, y- In the problem of Section 10, for instance, 
the adjusted values of the x coordinates of the n points were all 
required to be equal to a, which was then evaluated as x (Fig. 5) 
to Tnp.1rft the sum of tho squares of the residuals a minimum. 

So to take care of the general case we shall suppose that the 
adjusted values x,' and yi are subject to v conditions, to be symbol- 
ized as 


FHxi, X2, 

F®( 


I Vni 


a, b, c) = 0 
) = 0 


V equations 

for 

V conditions 


(3) 


F"( “ ) = 0j 

* The Bign 22 will denote Bummation over all observations, x and y both, if 
both are obeeived. 



[Ch. IV] THE GENERAL PROBLEM IN LEAST SQUARES 61 


The superscript on each F distinguishes that condition from an- 
other. Different sorts of problems are characterized by the dif- 
ferent kinds of conditions that the adjusted quantities Xi, yt, and 
a, h, c are subjected to. From the theoretical standpoint, the 
different problems are all conveniently handled alike. This is 
possible because there is only one principle of least squares, namely, 
the minimizing of 

Eqs. 3 ■wiU be referred to as the condiUona, or the conditicn 
egmiions. The functions F^, F^, etc., on the left, are the cmdir 
tion functions. They must be so chosen that when equated to zero 
they force the conditions that are to be imposed on the adjusted 
coordinates, angles, lengths, etc. 

The assumption behind this development is that the conditions 
would all be satisfied exactly by the true (unknown) quantities 
being measured, and the true parameters a, |8, 7, all of which, 
theoretically at least, could be had closely enough by increasing 
the number of experiments. 

26. Notation for the derivatives. The derivatives of the con- 
dition functions will be denoted by subscripts, as in Eq. 6 of 
Chapter III (p. 39). Specifically, the notation will be as follows: 


„ . aF* „ . dF^ 

F,i* = — > = . etc. 

dxi dyi 

„ . aF* „ . aF* 


Denoting differentiations by subscripts is very convenient in some 
work, as it is here. It is a common practice among mathematicianB. 

The subscript 0 in Eq. S below does not denote differentiation, but 
an approximation to the condition function F\ 


These derivatives, like the condition functions themselves, are 
functions of Si, X2, ■ • •, yn, 0, c. In what follows, we shall need 
numerical values of these derivatives, and fortunately, for most 
purposes, it will suffice to evaluate them with the observed quan- 
tities Xi, X 2 , • ’ ■, and with the best available approximationa 
Oo, bo) Co obtainable for the parameters, (cf. Ch. Ill; in particular, 
Exercise 13) . In othe r ] ^Qir (3ip.. ,FaL^ .i5. t0..bff.a-iiuiBbOT¥^iir^tt^ 
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MOEE complicated problems [Sec. 26] 

our best guess® at tbe numerical value of this derivative, and 
similarly for the other derivatives. 

We then write 

Fo^ = FHXi, Xi, ■ ■ •, Yn‘, 00 , bo, co) h=l,2,‘--,v (6) 

V equations 

Fo^ is a RTTinll number, being just the amount by which the con- 
dition F^ = 0 fails to be satisfied by the observed values Zi, 
Xs, • • and the approximations Oo, ho, Co- Similar statements 
hold for Fq^, Po®, • • ^ o- 

As stated earlier, ao, ho, co are approximate values of a, h, c. 

They oan usually be arrived at somehow, as by forcing three of 
the conditions, i.e., solving for the values of a, h, and c that make 
three of the condition functions vanish. This is the so-called 
method of selected points, concerning which more is said in the 
reduced type at the end of Section 65 (p. 138 ). Each Fe would 
be exactly zero except for errors of observation and the conse- 
quent impossibility of choosing ao, ho, cg to satisfy all the con- 
ditions simultaneously. 

26. The reduced conditions. Now let the conditions be made 
linear in the residuals 7aii, 7^2, • • •, 7j,ii, A, B, C, by expanding 
Eqs. 3 by Taylor’s series, retaining only the first powers of the 
residuals,^ and remembering that 

3 !,' = Xi 7 ^,^ 
yi=Yi~Yyi 

a = oq- A ■ (Calo’d = obs’d — residual) (6) 
b = bo -B 
c = Co - C 

^ If our best pess is too for wrong, a second adjustment will be required, 
but this iBiely happens in practice. See the quotation from Gauss on page 180. 

^ The problem of a streught line with no error at all in one of the coordinates 
(Exercises 1 and 7 in Sec, 66) is one in which there ore no squares and higher 
powers of the residuals to neglect, hence no discrepancies of the kind men- 
tioned (of. Bq. 3 of Ch. III). The simple example of the triangle in Chap- 
ters I and V is another. On rare occasions the residuals may bo ao large that 
the neglected terms invalidate the reduced conditions (Eqs. 7), in which event, 
in general, no systematic solution is available. An exception is the straight 
line under certain circumstances of weighting; see Exercise 6 of Section 06. 
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When this is done, the conditions originally expressed by Eqs. 3 
take the form 

L + Z FjV,i + FJ^A + Fi^B 

at 1/ 

+ Fc^C = Fq\ a = 1, 2, • • 

V equations (7) 

These are called the reduced conditions. They are equivalent to 
Eqs. 3, except for small discrepancies ariang from the neglect of 
higher powers of the residuals in the expansion. 

27. The method of Lagrange multipliers.^ Now if iS is at its 
minimum value, and if any or all of the residuals then undergo 
small variations (expressed by S), the variation in S will be zero 
to within higher powers of the variations in the residuals; in other 
words 

= £ wV&V = 0, one equation (8) 

The variations typified by &V are not arbitrary, but must always 

permit the residuals to satisfy the condition Eqs. 3, or their 
equivalent, Eqs. 7. So by differentiating Eqs. 7 we find that 

2 FjSV,i + 2 FyJ'SYyi + FM + FihB 

+ Fj'SC = 0, h = 1, 2, • • •, 

V equations (9) 

Now multiply Eq. 9 through by — Xa, an arbitrary multiplier, to 
get 

-Xft(2 Fj&V^i + 2 Fy,hVyi) - \hFM - Xi^bHB 

- 'KhFj'SC = 0, h = 1, 2, ■ ■ 

V equations (10) 

< This is the method of Lagrange multipliers; see his Mkanique andiy Ugue 
(1811), tome 1, p. 74; or Benjamin Williamson, Difereniial CaMus (Long- 
mans, 1893), Chapter 11. The least squares problem without parameters wan 
worked out by Gauss. He called his multiplier correHata, not mentioning 
T,B£r H.n E R . Many texts in least squares use the term “ correlatea " or “ cor- 
relatives " in this connexion, but apparently none makes any mention of 
Lagrange. The reference to Gauss is his Swpplemenlum Theoriae Combina- 
iimis ObaervalionvM Erronbus Minimis Obnosdae (Gottingen, 1826; Wsrhe, 
vol. 4), art. 11. 
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MOEE COMPLICATED PROBLEMS 


[Sec. 27] 


Add Eqs. 8 and 10 and collect coefficients of the variations S: 

£ ~ 22 b^viy vi ~ 

- - [\hFc’']SC = 0, one equation (11) 

In Eq. 11 there are two kinds of summations — there is the summar 
tion 22 running over all observations, and there is also the summa- 
tion over h, in which h runs from 1 to v, i.c., over all conditions. 
The latter summation will be denoted by the Gauss braekets [ ]. 

Here the number of parameters is taken os 3, If there were p 
parameters, there would be 2?i -f- p variations. For practice, 
the student should write out Eqs. 8-15 with (e.E.) » = 3 and 
» = 2 with two parameters. There is no other way to gain 
familiarity with the development. 

Eq. 11 contains 2n -f- 3 variations, 67*2, 67*3, ’ ’ 'i 57jn, 
SA, SB, SC. But on account of Eqs. 9, only 2ft -f 3 — v of these 
variations are arbitrary. Let Xj, X 2 , • • •, X„ be so chosen that v 
of the coefficients in Eq. 11 vanish; then tho coefficients of the 
variations in the remaining 2n + S — u terms must also vanish, 
because they are used with an equal number of variations, each of 
which is arbitrary. Then all the coefficients in Eq. 11 vanish, 
which means that 


Wxi 

ft equations; i = 1, 2, • • 

•, ft (12x) 

v«i = — lhM 

Wyi 

ft equations; i = 1, 2, • ■ 

•, ft (122/) 

[XfcPa'*] = 0 

one equation 

(13a) 

[X^f’s'^l = 0 

one equation 

(136) 

[XaFc*] = 0 

one equation 

(13c) 


Each residual (Fij or 7 „i) in Eqs. 12 is inversely proportional 
to the weight la*; or of the corresponding observation. Does 
this seem reasonable? It any observation is relatively infallible, 
having lo = «, then its residual is zero; i.e., there is no correc- 
tion. In curve fitting, for example, it sometimes happens that 
all the X coordinates are free of error; the corresponding resid- 
uals are then 0, and the calculated values of x ore tho same os the 
observed. 
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The V Lagrange multipliers (X) are no longer arbitrary; they 
now have particular values; they have been chosen so as to cause 
V of the coefficients in Eq. 11 to vanish (vide supra) . Their values 
can be found from Eqs. 13 and 15. We shall now derive Eqs. 15. 

28. The general normal equations. Now substitute (l/w*i) X 
[\hFzi'] for Vai, and likewise for y, in the reduced conditions 
(Eqs. 7). Collect the coefficients of Xi, X 2 , • ■ •, X„ A, B, 0, and 
in so doing set 


Lht 1 1 - • • 

Wxl 

Fyl^F ^ F 
1 ^ 1/1 




F fjn^F 


= LfcA (14) 

The following system of equations results. They may be called 
the “general normal equations.” For convenience, only the 
coefficients are tabled, the unknowns being written across the top. 
On the left of the equality sign, each coefficient is to be multiplied 
by the unltnown appearing above it, the plus sign between terms 
being understood. On the right, each Fq is multiplied by unity, 
hence the heading “ 1 ” for that column. 


ThB GENXBAL normal XqUATIONB 



Xj 

Xj ' " 


A 

B 

c = 

1 

^11 

L:i 

Lti • • • 

L,i 

Fa'- 

Fb' 

Fa' 

ft' 

Lit 

Ltt 

Ltt 

L,t 

Fa’' 

ft* 

Fa” 

ft* 

Lit 

Lit 

Ltt • • • 

Lyt 

Fa” 

ft* 

ft* 

ft* 

Lit 

Ltr 

Lt, • • • 


Fa* 

ft' 

ft' 

ft’ 


Fc^ 

ft* 

Fa* 

0 

0 

0 

0 

Fb'- 

ft* 

ft* ••• 

Fb* 

0 

0 

0 

0 

Fc^ 

ft* 

ft* ••• 

Fa' 

0 

0 

0 

0 


more complicated problems [Shc. 29] 


1 Alnnir the diagonal, h = k, and off the diagonal, 

. V coraring Bq. 14 with Eqs. 8 and 9 of Chapter III 

ft in be sL thS; lJb the weight of the condition 
iiJiwrfitting it is in fact BometimeB useful to write l/1f m 
nlaoe of L, as will frequently be done latM. (Cf . also Remark 3 
SnTl35.) ThetermL^^offthe di^on«li8 the reciprocal of the 
product variance of the two condition functions r and F . It 
S Ls be observed that the diagond in the general normal 

ikns is made up of the variances of the con 

werv term of which is positive, and that the terms oft the 
Snal are product variances, which can Bometimes be 

g. Since Lhk = hck, as is indicated by Eq. 14| the 
ooeflaeients of the unknowns in Eqs. 15 are sy^etrical about 
the diagonal. Because of this symmetry, it will be possible, fol- 
lowing Gauss, Doolittle, and others, to shorten the numerical 
coi^tation for finding the unknowns (Secs. 34 and 61). In 
the abbreviated solution, it is not necessary to enter the coeffi- 
oienta below the diagonal (see Seo. 30). 


The general normal equations are v -|- 3 in number, and can be 
solved for the v + 3 unlcnowns written across the top. Special 
methods of solution wiU be taken up in Sections 34 and 61, but for 
the present we shall only note that once the residuals A, B, and C 
are-found, the final (adjusted) values of the parameters are ob- 
tained by subtracting the residuals from the approximate values, 

as shown in Eqs. 6. . , , , 

The solution of the general normal equations yields also numeri- 
cal values for the Lagrange multipliers Xi, Xg, - • X„ which through 
Eqs. 12 enable the residuals (7) to be calculated. The observa- 
tions Xi and Yi are then adjusted by subtracting the residuals, 
again according to Eqs. 6. The adjusted quantities Xi, along vdth 
the adjusted parameters found by Eqs. 6, will satisfy the v condi- 
tions expressed by Eqs. 3 (p. 50), or their equivalent, the reduced 
conditions, Eqs. 7 (p. 53). 

Exerme. Apply Taylor’s series to any one of Eqs. 3 to derive 
the corresponding reduced condition shown as Eq. 7 . 

29. Short expression for S. The normal equations are really 
normal. The matrix of the coeflScients is positive definite. By 
tipfinitirni , S = 'Lw re^. Now by substituting for the residuals 
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in terms of Eqs. 12a; and 12y, we find that 

a Wxi 1/ Wyi 

= + \2Eii,i® + . . . XyP^i")^ 

Wxi 

+ — 0^lPx2^ + X2P«2“ + • • ■ + 

Wx2 

+ . • • H (XlE yn^ + \zF j,n® + ‘ • ■ + \fF ynY 

Wyn 

= LiiXi® + + • * • + L,f\y^ 

+ 2 (Z/i 2 XiXj + the other cross-product terms) 

Another way of writing this is to border the following symmetrical 
array. 



Xi 

Xa 

Xa 

• • • ' Xjr 

A 

B 

C 

>^1 

All 

Ail 

Aai 

• • • A,i 

Fa'- 

Fgl 

Pa'- 

^2 

Lit 

Lit 

Agi 

• • • Lpi 

Fa" 

Fi" 

Fa" 

^a 

An 

Aig 

Agg 

- Lpi 

Fa" 

Fi" 

Fa" 

i 

« 

• 

• 

• 

• 

■ 

• (16) 


Ai, 

Lip 

Agp 

... L„ 

Fa* 

Fb* 

Fa' 

A 

Fa'- 

Fa^ 

Fa" 

... Fa> 

0 

0 

0 

B 

Fi' 

Pg* 

Fg’ 

... Fft' 

0 

0 

0 

a 

Fa'- 

Po® 


... Fa* 

0 

0 

0 


= Xii^’o^ + ^2^0^ + XsFo^ -!-••• + y^pPa” = [XfcEo*] 
by Eqs. 15, page 65. We have thus discovered that 

S = [Xa^’o'"] (17) 

In this way, S, the minimized sum of the weighted squares 
of the residuals, is expre^ible in terms of the Lagrange multipliers; 
wherefore, so far as ;S is concerned, it is not necessary to compute 
the residuals and square them. Later, we shall see that 5 can be 
computed by a systematic procedure without even finding the 
Lagrange m^tipliers (Secs. M and 61). It is a fact that in some 
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problems it is nevertheless advisable to compute the residuals, so 
that th^ can be examined individually (of. Sec. 78). 

Gauss derived Bq. 17 for the case of geometric conditions, for 
which parameters axe absent. For other special expressions of S, 
useful in curve fitting, when parameters are present, see the exer- 
cise following this section; also Exercise d, pa^_ 163. 

Since S > 0, the quadratic form (16) is positive defimte ; that 
is no matter what values be given to Xi, X 2 , • • ■, X„ A, B, C, lie 
oiadratic form (16) can not be negative. The symmet^ of the 
general normal equations (Sec. 28) has already been noted; hence 
these equations are really normal -i.e., they are not only sym- 
metric, but the quadratic form of the coefficients is positive 

definite. 

Exercise. 


-S = 


I'll 

L21 

Lai • • 

Lpi 

Fa^ 

Ft^ 

Fa' 

Fo' 

Li2 

Liz 

L 82 • • 

Lp2 

Fa® 

Ft® 

Fa® 

Fo® 

Liz 

• 

I '28 

• 

Las • • 

c 

Lps 

F„® 

Ft® 

1 

Fa® 

Fo® 

Li„ 

Lzy 

Lap * • 

Lpp 

F„” 

1 

Ft” 

Fa” 

Fo” 


Fa^ 

Fa® •• 

Fa- 

0 

0 

0 

0 

Fi^ 

Fz^ 

Fa® •• 

Fz^ 

0 

0 

0 

0 

Fo^ 

Fc^ 

Fc® 

F/ 

0 

0 

0 

0 

Fo' 

Fo= 

Fo® •• 

Fo” 

0 

0 

0 

0 

I'll 

L21 

L31 ' • 

Lpi 

F„' 

Ft' 

Fa' 


Li2 

L22 

L32 • • 

L„2 

F„® 

Ft® 

Fa® 


Liz 

Lzz 

Las • • 

L,8 

F„® 

Ft® 

Fa® 


Liv 

Lzv 

Lzv ' 

• Lpp 

. . 

Ft” 

Fa” 


Fa^ 

Fa^ 

Fa® ■■ 

• Fa” 

0 

0 

0 


Fi^ 

Fi? 

Fi,® •• 

• Ft” 

0 

0 

0 


1 

F^^ 

Fa® •• 

• Fa” 

0 

0 

0 



8 Maxime B6oher, Higher Algebra (Maomillaa, 1907), page 160. 



Part C 

CONDITIONS WITHOUT PARAMETERS 


CHAPTER V 

GEOMETRIC CONDITIONS 

30, Adaptation of the general solution to conditions without 
parameters. When the conditions imposed by Eqs. 3 or 7 (pp. 60 
and 53) of the last chapter are pometric, there are no parameters 
or adjustable constants. The quantities o, h, and c then do not 
exist, and in the general normal equations (p. 65), all the rows and 
columns containing A, B,C, Fa, and P/ are to be deleted. 
The Lagrange multipliers (X) are the only unloiowns left, and only 
the square array of L coelBcients remains. The general solution 
thus reduces to Eqs. 1 shown below. 


Xl 




» 1 


Lu 

hi 

hi 

• • Lie 

Fo'- 



hi 

hi 

• • Li, 

Fo^ 




hi 

■ • hr 

Po’ 

(1) 




hr 




Here the coelhcients below the diagonal have been omitted, since in 
the abridged solution soon to be learned, those below the diagonal 
are not used. The coefficients are to be read " down to the diagonal, 
then to the right.” The unknowns are the v Lagrange multiplierB. 

This type of problem (no parameters) was solved by Gauss,^ 
and is treated satisfactorily in many textbooks. It arises in 

* See the reference to Gauss in Section 27, page 53. 
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CONDITIONS WITHOUT PARAMETERS [Sue. 31] 


geodesy, surveying, and in astronomy, and this accounts for the 
attentions of Gauss, Bessel, and Encke, who were mainly interested 
in the problems of adjustment arising in astronomy. 

31. Example: the plane triangle. We shall return now to the 
triangle problem discussed in Section 3 (see Fig. 3, p. 7). The 
angles are measured with a transit. The weights might arise from 
the number of repetitions on each angle. 

Observations: Xi, X%, Xz 

Weights: toi, w^, ws 

Calc’d values: Xu xq, X 3 (to be found) 

Here there is only the one condition, namely, 

*1 + aJa + aja = 180® (This corresponds to Eq. 3, p. 50.) (2) 
so we write 

^(®i) ®a, ^s) = »i + + *3 ~ 180° (3) 

(There is only the one condition, so no superscript on the F is 
needed.) This condition function F will be zero when we are able 
to insert the adjusted values a:i, X 2 , X 3 into it. By inserting the 
observed values we calculate 

Fq = + .^2 ~ 180° (See Eq. 6, p. 62.) (4) 

Fg is not zero unless Zi + Za + Za happens to be exactly 180°, 
in which case no question of adjustment arises. The derivatives 
of E are 

El = = 2?3 = 1 (See Eq. 4, p. 61.) (5) 

There is only one L coefficient (why?). It oould be called Ln 
but no subscript is needed, so we shall use simply L. It is cal- 
culated as follows: 


T- ^iFi , F 2 F 2 , FgFa 1 , 1.1 

h 1 1 

Wl W2 Ws Wl W2 W3 

There is but one normal equation, namely, 


(Eq. 14, p. 66) (6) 


Lh = Fg 


( 7 ) 
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X = 


Fa 

L 


( 8 ) 


The numerator, Fq, is the amount by which the observed angles 
fail to dose. The denominator, L, is l/wi + l/w 2 + l/icsj which 
happens to be equal to 1/wp by Eq. 9 on page 40 (propagation of 
mean square error). 

After X is worked out numerically, we may find the three resid- 
uals by Eq. 12, page 54: 


T"i = -XPi = - 

Wi 

V2 = —\F2 = - 

W2 W 2 

78 =-X?’3 = - 
VI3 % 


( 9 ) 


The adjusted angles are then 1 

xi = Xi — Vi] X 2 = X 2 — 1 ^ 2 > Xz = Xtt — Vs ( 10 ) 


The sum of the adjusted angles is identically 180°, for 

*1 + *2 + ais =-^1 + .^2 + ~ (1^1 + 1^2 + ^s) 

= Xi + X2 H-'Xa 

_ fl 4. 1 + 

\«)1 103 ~ Ws/ 4, Jl 4_ _L 

Wi W2 Ws 

= 180° exactly , (11) 

The equations for Vi, 72, and Fa are valid no matter how 
large Fo is. This is a case whei’e there are no higher powers of 
the residuals to be neglected, and is in contrast with the more 
general statement in footnote 4, page 62. 

Note that the residuals ore inversely proportional to the 
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weights of the observationfl; that is, 


7l . Fj • Vj 


(12) 


Thus, in this problem, the adjustment by least squares simply 
t ftVaB the excess or deficiency Ft (which will ordinarily be a small 
amount, perhaps a few minutes of arc) and distributes it among 
the three angles in inverse proportion to their weights (cf. Ch. I, 
p. g). The student should reflect on this at length. If the 
action of least squares seems reasonable in this simple problem, 
it may be so in more complicated ones, even if we are not so 
easily able to visualize its working. Even in more complicated 
problems, the principle is the same (the minhnizing of 2 res® 
or of x’) ! if* is only the conditions to which the adjusted values 
are subject that d&er from one problem to another. 

Exercise 1. Stow that the condition 

= 180“ (2) 

determines a plane distant 180“/V' 3 from the origin, and cutting 
equal intercepts from the axes. The calculated point lies on the 
plane, and the observed point off it. If the weights are all equal, 
the distance between the observed and calculated points is to be 
minimized, in which case the line segment joining the observed and 
calculated points is perpendicular to the plane ®i + ®2 + *3 = 
180“. See Fig. 11. 

If the weights of the observed angles are unequal, the distance 
between the observed and calculated points is not to be mini- 
mized, but rather the quantity 

+ W3(Z2 - xgp + w,(Z, - Sj)» (13) 

Exercise Any possible plane triangle is represented by a 
point on this plane for which x\, Xi, and ®3 are positive. Any 
method of adjustment would consist of picking off some point on 
this plane, corresponding to a given observed point Xi, Xi, X 3 off 
the plane. 


Exercise 3. Solve the triangle problem (p. 60) without the 
Lagrange multiplier. 

Hint: Take <8 = 2 wV^ ^T,w{X -x)^ 


(14) 
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Eio. 11. The three angles of a plana triangle constitute the coordinates of 
a point. The calculated (or adjusted) angles add to 1S0°. The point repre- 
senting the calculated angles lies on a plane distant ISOVV's from the origin. 
The obaenred point lies beyond the plane if there is an observed excess beyond 
180°, but lies on the under side of the plane if there is an observed deficiency. 
It lies on the plane only by accident, in which case no adjustment is required. 

By the one and only condition on the adjusted values, we may taJce 


Is = 180° — ii — I 2 

(15) 

H 

0 

1 

Na 

1 

(16) 


where, as before, 

Fo=Xi + X2 + Xs- 180® 

Then 

S = wiVi^ + + WiiFo -Vi- Vi? (17) 

*1 and 312 are independent; so axe 7i and 72- Hence we may 
set dS/dVi and dS/dVi both equal to zero. The result is 

wiVi — wsiFo — Vi— Vi) = 0^ , . 

^2Vi-v>a( “■ ) = .0J 

It follows that 


wiVi = 1V2V2, 


( 19 ) 
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and that 


1 


T^i = rr 


“il + l + i- 

loi iDi wa 




^a = :r 


+ 1. + i- 

Wi Wa Wa 


Va = ^ 


Wa 1 111 

1 1 . 

loi Wa Wa 




( 20 ) 


which ore equivalent to Eqa. 9 on page 61, obtained with the 
Lagrange multiplier. 

All problems in least squares con theoretically be solved with- 
out the use of Lagrange multipliera. Occasionally it may even 
seem easier to dispense with tiem, but moat problems then be- 
come hopelessly involved, as Kunraiell discovered.* 

32.' The plane triangle continued. The weights of the adjusted 
angles, and any function of tiiem. Keturning to the triangle 
problem of the last section, suppose we ask for 


and 


The weight of angle %x 


The wfflght of the sum of *1 -j- sa + * 3 , after adjustment 

Of course we know in advance that the weight of this sum must be 
infinite, since we forced it to be a definite amount, 180°; but it 
will be interesting to see if this r^ult comes by the routine about 
to be described. The rules for finding the weights of functions of 
the adjusted observations are illustrated in what follows, and a 
more complicated example will be worked out in the next chapter. 
The theoretical proofs will be found in several books on least 
squares, for example, 0. M. Leland’s Prodded Least Squares 

* Charles H. Eummell, The Analyst (Des Moines), vol. 6, 1879: pp, 97-lOB. 
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(McGraw-Hill, 1921 ) and T. W. Wright and J. F. Hayford’s 
Adjustment of Observations (Van Nostrand, 1884, 1906). 

Let 


II 

(21) 

= X1 + X2 + X3 

(22) 


CP- and G® are then the functions whose weights are wanted. As 
many more functions could be added as desired, but here we shall 
be content to see just the weights of xi and of xi -1- 12 + X3 worked 
out. The procedure is as follows. We need to form certain sums, 
and to this end we make up the following table, numerical values 
ordinarily being inserted in place of the symbols in the body of 
the table. F is defined by Eq. 3 on page 60 . 


(1) 

i 

(2) (3) (4) 

Fi GP QP 

(S) (6) (7) 

Ft op GP 
y/m Vwf 

(8) 

Sum 

1 

1 1 1 

_l_ _l 1_ 

Viffi Vv>i Vifli 


2 

1 0 1 

J- 0 J- 

1/W2 \/v)i 

for numerical 
check 

3 

1 0 1 

1 0 1 
y/wt y/VlB 



Next step, from columns 5 , 6, and 7 the sums called for in the 
nArmn.1 equations can be evaluated, as shown below. 



as defined on page 60 


ft (I U (t tl 


[ ] TnAnriH summation, as in Section 27 (Gauss’ notation). These 
sums are appended in the and C® columns, and the solution 
proceeds. 
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In a numerical solution, a sum column would bo introduced at 
the right for a check, and the spaces filled in by the ellipses would 
be filled in with numbers (see the note in reduced type on p. 33), 
For a numerical illustration see pages 82 and 83. 

Row I gives X = Po/L, as already foimd on page 61. Looking 
next at the “ 1 " column in Row II we see —PoFo/L, which has the 
value - XFo, and which by Eq. 17 on page 57 is none other than 
- S. Thus, Z is computed in a routine manner without first 
finding the individual residuals Vi, V 2 , and F 3 . 

The variance coefficient of G^, or the reciprocal of its weight, 
appears in the column of Row and the variance coefficient 
of CP, or the reciprocal of its weight, appears in the column of 
Row n®. 

Befm adjustment, the weight of G^ was lOi, the weight of the 
observation Zi: o/ter adjustment, the reciprocal of its weight is 
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1/wi — XjlMx. Now of course 

1- J- 1 

toi Zaoi* Wi 

which means that the weight of xi is greater than the weight of Zi. 
That is, the weight a^ter adjustment is greater than the weight before, 
which seems reasonable enough; the observations on the other two 
angles help to estimate xi, and to increase our confidence in its 
value. After the adjustment we feel that we know more about 
the triangle than before. 

In parUeular if all three angles have the same weight before 
adjustment, then if = u )2 = ws = 1, the weight of ®i after 
adjustment is l/[l/wi — = 1/{1 - 1/3} = 1.6, which 

is 60 percent peater than the weight of X\. The adjustmmt 
therefore increases the weight by 50 percent. The same thing is of 
course true for the other angles. 

If the weight of an angle has been increased 60 percent, its 
standard error has diminished 18 percent, since 

= (S.E, afhr' (See Eq. 16, p. 22.) 

Next consider the weight of xi + X 2 + From Row II® in 
the form above we see that the reciprocal of the weight of this 
function is zero; in other words, the weight of + ®2 + 23 is 
infinite; it is therefore known absolutely. The adjustment 
forced Uie sum to be ISO”, and it is no surprise to find its weight 
after adjustment to be infinite. 

This simple example gives a glimpse of the method for the solu- 
tion of problems involving rigorous conditions. A guide for 
systematic computation, and a more complicated example, are 
given in the next section. 


Exebcibss 

Exercise 1. Tak;e the values of 7i, ^ 2 , and Fs found earlier, 
namely X/ioi, X/mi 2 ) and yws, and show by direct substitution that 
XFo, the negative of the extreme left entry in Row II of the tabular 
tion shown above, is actually S, or loiFi® W 2 V 2 ^ + icsFa®. 
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Exo'cise H. By the use of Eq. 7, page 40, show that the variance 
of the sum si + *3 + xs of the adjusted angles of a plane triangle 
is 0; hence its wei^t is infinite. 

HvnU xt = 180“ - a:i - ®3 


whence 


+2r2so'!ffi 

(o) 

Cl® = ffj® + Vs® + 2ratatat 

(6) 

ffa* => ffi* + o'j® + 2ri8(riO’j 

(c) 

ffj® = ffi® + ffa® + SrisCifri 

(d) 


By nnmhiTiing the last three equations with Eq. a it is found 
that 

Var (ii + xs + a:s) = 0 (e) 

Exercise S. Observations Xi, Xg, • • *, Xn, with weights wi, 
W 2 ,'”, w„ are taken on n quantities, the adjusted values of which 
are connected by the one condition 

®i + a:2 + • • • + = O' 


By making use of the scheme outlined in Section 32 for finding the 
weight of a function after adjustment, show that the weight Ur of 
the sum + xs + • • • + »" < n, after adjustment, is 


where 


and 


1 = -L_X 

Ur Wr Wr‘ 


-L = i + i + ... + i 

Wi 102 'Wr 


^ = - + -+-" + - = L 

W Wi W2 Wn 


In particular, if loi = i 02 =»••• = io„ = 1, 


C/,= 


n 

r{n — r) 
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If n = 3 and y - 1, 17, = 3/2, which is the special case of one 
angle of a triangle, already worked out on page 67. If w = 4 as 
for a quadrilateral, one angle, qffer a^ustmmt, has the weight 
4/3, the sum of any two angles has the weight 1, and the sum of 
three angles has the weight 4/3. 


This problem has application also in the social sciences, where 
proportions are observed by sampling methods, and the total 
count is known from other sources (Ch. VII). For a cell that 
is not too small (i.e., for one having a sample frequency of pos- 
sibly 10 or higher), the weight of the observed frequency may be 
assumed inversely proportional to that frequency, and the 
variance thereof equal to the cell frequency. 


a two-ceUed table, the total count of the two cells beinir known 


If Wi + W 2 “ n, then ni/n and wj/n are the observed propo^ 
tions. Denote them by p and g. Thenp + ge= 1,Z(= l/wi + 
l/wj = ni + n 2 = n, and the weight Ui of the cell ni after 
a^tment is given by the equation 


“ = variance of jii = — 
uii 



= m - -- = mg = Jipg 


Thus the variance of is reduced from m to mpg by the adjust- 
ment. The variance of the proportion p is reduced from p/m to 
pg/m. The ratio of the variance after adjustment to the vari- 
ance before adjustment is thus equal to g. The reduction in 
variance is considerable when g is small, i.e., when p is nearly 
unity, as happens when wi is nearly all of n, 
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SYSTEMATIC COMPUTATION FOR GEOMETRIC 
CONDITIONS 


33. Steps in the fonnatlon of the normal equations. There 
will be observations, weights, and conditions imposed on the 
adjusted values. 

Observations: Xi, X 2 , • • Xn 


Weights: isi, W 2 , • * Wn 


Conditions: la, • * ■, ®n) = 0 

X2, • • •, In) = 0 

F\xi, X2, • • •, x«) = 0 
F*(xi, X 2 ,-",Xn) = 0 ^ 


(These are Eqs. 3, 
p. 60, except that (1) 
here there are no 
parameters.) 


hi step. Write down the conditions, i.e., select the appropriate 
F functions. Decide also on the G functions whose weights are 
wanted. One then works out the values of Fq, which will usually 
turn out to be small numbers, since the conditions will be nearly 
but not quite satisfied by the observations; see for instance 
page 77. 


By the uae of tiie Tedptooel matrix as explained in Section 36, 
one need not decide on all hie G functions at ^e start; more con be 
added later without great inconvenience. 


The solution will be illustrated with four conditions; i.e., the 
number v in Eqs. 3 on page 50 is taken as 4, which will be the 
number of Lagrange multipliers (X). Expansion or contraction to 
more or fewer conditions is easy. (In the simple triangle problem 
of Sec. 31, p. 60, there was only one condition, and one X.) 

We shall assume here that we want to find the weights of two 
functions of the adjusted values. Let these functions be desig- 

70 
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oated as 

G^ixi, Xi, • • •, Xn) 

Xi, ■ ■ Xn) 

(In the triangle problem of Sec. 32, was Xi, and G^ was 
Xi + Xi + Xi-, seep. 65.) 

2d step. This requires some differential calculus. It consists 
of writing down the various derivatives that are needed, such as 

E. 1 E. 1 


Fi^ or ^ 


Fi^ or — . 


Fi^ or 


These are used for forming the L coefficients, according to Eq. 14 
on page 55. We shall also need the derivatives of the Q functions, 
such as 

Gi^ or “I Gi^ or — - 

dzi 0X2 


Gs* or 


G 2 ® or — 


which are to be used in computing the weights of the G functions. 

Sd step. Work out the numerical values of the derivatives ; see, 
for instance, page 78. In each case, the observed values Xi, 
Xi, • • ‘fXn are used in place of the adjusted quantities xi, Xi, ■ • •, 
Xn, since approximate values of the derivatives are usually close 
enough; at least they will have to suffice till we can get better ones. 
The following table is made up, numerical values being inserted in 
the spaces. Naturally, more or fewer columns will be needed in 
various problems, and different computers wiU work differently 
even on the same problem. The layout will also vary, depending 
on what type of calculating machine is available. Only general 
directions can be given in advance of a specific problem. 
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TABLE 1 {3d step) 


% 


f.i Gj® 

1 

Sum 

1 

■ 1 • • • • 


. . . 

2 


1 . * 4 * • 


n 

f • 

• • » • • • 

... 


The sums at the right in Table 1 are formed oxcliialvo of the entries for the 
weights. They ore useful in cheeking the formation of Table 2. 


4th Step. Fonn Table 2, which is derived from Table 1, by 
multiplying the F and 0 derivative by the corresponding values of 
as indicated in the headings of Table 2. 

TABLE 2 


The uatrix iok the tormaoton op the mormal equations (4th step) 



Table 2 is termed a matrix because from It is formed the normal equations. 
Moreover, in matrix notation, the formation of the normal equations is the 
.product M'M, M being the matrix of Table 2, ond M' its “ transpose.” 

The sums ^own at the right and across the bottom of Table 2 are used for 
checking the formation of the normal equations. The sums themselves are 
checked by adding them down and across, to see that they add to the same 
grand total either way (the “ comer check "). 

There are various procedures tiiat one can follow in computing Table 2 
from Table 1. With automatic multiplication, the computer may prefer to 
use 1/ y/wi as a constant factor in row i, reading off the individual products 
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(FiVVw* ®to.) aad entering them in Table 2 , cumulating the sum + 
Fi* + Fi® + Fi® + of ffie multiplieiB to check with the aiima already 

entered in Table 1. 

TOth a machine having two multiplier registers, one for cumulating 
the quotients, and the other for reading individual quotients, the computer 
may cumulate a sum in either the horiaontal or vertical without extra effort; 
If the machine moreover permits the dividend to be altered independently of 
the keyboard, one may set Vni< on the keyboard and use it for a divisor 
throughout an entire horizontal row of Table 1 , entering the individual quo- 
tients in Table 2, and at the same time cumulating the sum Si to be entered 
at the right. 

The use of punch card equipment for forming normal equations may save 
time and expense on large projects. 

step. The coefficients in the normal equations are now to be 
formed from Table 2. By recalling the definition of Lu in Eq. 14 
on page 65, and by introducing the and columns for the 
weights of the Q functions as used in Chapter V, we may rewrite 
the normal equations of page 69 in the form shown below. 
It will be observed that the terms on the diagonal are sums of 
squares formed from the columns of Table 2, and that the terms off 
the diagonal are the sums of cross-products formed from these 
columns. The numbers entered in the and C® columns are like- 
wise the sums of squares and cross-products. 


Normal dquationb 

Vnhnowna 


Row Xi Xa Xa X 4 1 ^^1 Sum 
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The sum column at the right checks the formation of the normal 
equations. Herein are entered (in pencil) tho cumulation of the 
cross-multiplications formed with the sum column of Table 2 ; these 
should agree with the sums of the terms in the normal equations, the 
“ 1 " column excluded) see Table 3 in Section 34, and the check 
formed Immediateiy below. If no errors are found, the sums entered 
in pencil at the right of the normal equations are altered to include 
the " 1 " column, and the solution proceeds, being checked at the 
pivotal points (see the check marks in Rows II, III, and IV of 
the numerical solution in Sec. 34). Tho sums and 

[CriWMl must be checked otherwise, os by repetition. 

The 0 in the bottom row of tlie normal equations is appended for 
the computation of the minimized sum of squares, S. The columns 
and assist in the computation of the weights of tho functions 
and 0^. 

The solution oi the equations is to be carried out by the routine 
process aheady seen in simplified form on page 66, and to be illus- 
trated more fully on pages 82-83, and symbolically on page 168. 
When the numerical values of the Lagrange multipliers (X) have 
been worked out, the residuals Fi, • • F„ are to be calculated by 
Eq. 12 on page 64, and then used to find the “ adjusted observa- 
tions " * 1 , X 2 , • • •, as follows; 

®i “■Xi ~ Fi = XjF 1® -|- X3F1® -j-X4i'’ 1^) 

®2=JPa“"F3=X2— (XiF3^*HX2F3®-1-X3F2®4'X4F2*) 


»„=Zn-F„«Z„- 1 

It should be noted that the numerical values of the derivatives 
I Pi^) etc., required in the parentheses, are ready for use in 
Table 1, p. 72. 

34, Numerical example: a surveying problem. A surveying 
party measures the sides and angles of the plane triangle PQB, 
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witii the following results: 


On angle P; 51“ 06' 
(4 observations) 08 

05 

06 


Average 51“ 06'.26 

On angle Q: 

96“ 05' 

(2 observations) 

04 

Average 

95“ 04'.5 

On angle R: 

33“ 49' 

(2 observations) 

50 

Average 

33“49'.5 


Sidep: 1723.7 ft. 
“ g: 2206.4 “ 
“ r. 1232.7 “ 


fk 



of this plajie triangle have been 
measured. The sum of the 
adjusted angles must be 180°; 
and the adjusted angles and 
sides must satisfy the sine lavr. 


The transit man, from previous experience, has reason to believe 
that the standard error of single measurements on one angle is 
about one minute of arc, or 0.00029 radian. He takes the standard 
error of the chainmen to be one foot in 10,000 feet, and in propor- 
tion to the square root of the distance chained. The weights of the 
observations on the angles and sides are then in ratios as follows; 


u,p :u,p 


0.00029=* 



( 4 ) 


These ratios come from Eq. 13 on page 21, wherein the weight 
of a function / was defined to be inversely proportional to its 
variance. Since weights are relative and not absolute, the factor 
of proportionality (o-^) in Eq. 13 on page 21 is arbitrary and can 
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be chosen for convenience; accordingly we let 

(T® = I 0.00029® = 423 X KT® (5) 

whereupon the wa^ta take these simple values; 

Wp “2, ioq = 1, wr = 1, 

Wp * 24,6Xl0rB, la, = 19.2X10-®, lOr = 34.3X10-® 

It should be noted that the final adjusted values of the sides and 
angles, also their standard errors, are in no way dependent on the 
arbitrary choice made for j®; if v® is doubled, all the weights are 
also doubled, and the standard errors of all functions are left 
unaltered. Likewise x® is unaltered. 

The solution of the problem proceeds now according to the steps 
outlined at the beginning of this chapter (Sec. 33). 

fsf step. The adjustment must be carried out to enforce the 
following three geometrical conditions: 

sinP _ sin Q _ sinfi 
p q r 

P + Q + B*180’ + e (8) 

6 bang the spherical excess, which, owing to the small size of the 
triangle, will here be taken as zero. If it were other than zero, Fq 
(Eq. 10) would be altered by the amount e, and tho adjusted values 
of the sides and angles and their standard errors would all be 
affected in an obvious manner. 

For forcing the three conditions, let us set 

PHP,Q,R,V,q,r) = ^-^^ 

P Q 

(I II II II in _ P _ sin R 
p r 

^(" “ “ '“"') = P + Q + 12-180“ 

(The numbei of oondltione is 3; i.e., the number 1 / occurring in 
Eqs. 3 on page GO is 3.) 
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f when evaluated do not give zeros, but give the nmnll 

numbers Fq^, Fq^, which by direct substitution are found to be 

, sin 61“ 06'.25 sin 95“ 04'.5 

® ” 1723.7 2205.4 

= -1.3271 X 10“^ 

, sin 51“ 06'.25 sin 33“ 49'.5 

“ 1723.7 1232.7 

= -0.5416 X 10“’ 

F^ = 51“06'.26 + 95“04'.5 + 33“49'.6 - 180“ 

= 0“ 0'.25 = 7.27 X 10-® radian 

If it had happened that the observations satisfied the conditions 
exactly, then F^, F^, and F^ would have turned out to be zeros, 
and the adjusted values would have been identical with those 
observed. As it is, the observations satisfy the conditions nearly 
but not exactly, i.e., Fq^, Fq^, and Fo* are small but not zeros. 

Fo® is the amount by which the sum of the angles exceeds 180“. 
In the simpler problem wherein the sides were not measured {vide 
mpra, Sec. 31) it turned out that the least squares adjustment 
was simply an apportionment of this discrepancy among the three 
angles in inverse proportion to their weights. Now, however, the 
sides are involved; wherefore the adjustment, though possibly as 
reasonable as before, will not be so easy to arrive at. By looking 
ahead to page 84 we see that, in contrast with the residuals on 
page 61, the adjustments on the angles will not now be all m the 
same direction. 

Now suppose that for some reason or other we should like to 
know the weights of 

Angle P 

The sum P + Q + It 

The area of the triangle, which may be expressed as |pr sin Q 

Any number of others could be added (at increased labor) but three 
will suffice here. For those just named we take the three G 
functions 

(?i = P, (^ = P + Q + R, G* = iprBinQ (11) 
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1 step. The derivatives of the F functions are 
cos P 




P 


JPpH = ppZ = 1 


1 COS Q -n Q n Q 

Fq^ Fq^ = 0 Fq^ = 1 


Fb^ = 0 


_ 9 cosiS „ , 

Fe^ — Fn^ = 1 


^ p2 

pa. sinP 

Pp® = 0 

p 1 “qQ 

^4 - g2 

F,^ = 0 

II 

o 

= 0 

2 _ sinB 

pa 

11 

O 


( 12 ) 


(13) 


The derivatives of the G functions are 
Gpl = l (?p2^1 ffp3=0 

The other Gq^ = 1 G*g®* = cos Q 

five derivatives Gr^ = 1 Gr^ = 0 

are zero The other Gp® = f r sin Q 

thi’ee derivatives Gj® = 0 
are zero Gr® = ip sin Q 

Sd step. The nearest numerical approximations that we can 
produce for these derivatives are found by substituting the 
observed angles and sides into the expressions just worked out, 
and these approximations will be more than close enough. 


TABLE 1 


Tnn DBBIVATIVBB (3d step) 


i 


lO'P,! lO'Fi* 

Gi^ Gi^ 0(8 

p 

Q 

R 

V 

s 

r 

2 1.41 

1 1 

1 1 

24.6 • 10-8 4,90.10-4 

19.2 " 4.38 “ 

34.3 " 6.86 " 

364 364 1 

40.1 0 1 

0 - 674 1 

-0.262 -0.262 0 

0.206 0 0 

0 0.366 0 

11 0 

0 1 -93916 

0 1 0 

0 0 613.9 

0 0 0 

0 0 868.6 
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step. \/io is now used as a divisor to form Table 2 from 
Table 1. 


TABLE 2 


The MATBnc for the formation op the normal botjations (4th step) 


i 


10»Fi® 

ft® 


G^ 


Bum 

y/Wi 

y/wi 

y/Wi 

y/Wi 

y/V){ 

y/Wi 


p 

0.267 

0.267 

0.707 

0.707 

0.707 

0 

2.635 

Q 

0.040 

0 

1 

0 

1 

-0.094 

1.646 

B 

0 

-0.674 

1 

0 

1 

0 

1.326 

V 

-0.628 

-0.628 

0 

0 

0 

1.238 

0.182 

i 

0.468 

0 

0 

0 

0 

0 

0.468 

r 

0 

0.625 

0 

0 

0 

1.466 

2.090 

Sum 

0.237 

-0.320 

2.707 

0.707 

2.707 

2.600 

8.647V 


The powers of 10 in Table 2 are chosen with regard to conven- 
ience, and to bring the number of decimals to uniformity from 
column to column, to facilitate the cumulation of squares and 
cross-products in forming the normal equations (the next step). 

At this stage one may also cut off superfluous figures, reserving, as 
a rule, not more than three or four in the largest number occurring 
in any one column. This often means that some other entries in 
the same column appear as zeros, but this is as it should be. 

J 

Bth step. The cumulations of squares and cross^-products from 
the columns of Table 2 provide the coefficients required for the 
normal equations (Eqs. 2, p. 73). For instance,^ 

10® [^] = 0-2572 + 0.040* + 0* -f- 0.628* 

+ 0.468* + 0* = 0.565 (14) 


as seen under Xi in the normal equations. Also 


10® = 0.257 X 0.707 + 0 - 0.674 + 0-i-0-f0-b0 

= -0.492 


^ The Bubsoript t will be omitted for convenience occasionally. 


( 16 ) 






ra-o- 


* Cksek (Powers of 10 are disregarded in the sum checks) : 

0.566 + 0.346 + 0.222 + 0.182 + 0.222 - 0.667 = 0.879 
0.346 + 1.190 - 0.492 + 0.182 - 0.402 + 0.262 = 0.096 
0.222 - 0.492 + 2.600 + 0.600 + 2.600 - 0.094 = 6.136 

The sums formed below the table to provide a check do not 
agree exactly with the numbers starred in the table, which are 
formed with the sums s,- of Table 2, but the agreement is within 
errors of rounding off, whereupon we conclude that the arithmetic 
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in Table 3 is correct, save for the three [CrCr/io] sums, which must 
be checked independently, as by repetition in reverse order. The 
cumulations shown in Table 3 are then entered into Rows I, 2, 3, 

4 of the tabular scheme for the normal equations on the two fol- 
lowing pages. The numbers entered in the “ 1 ” column of Rows 
I, 2, and 3 come from the valu^ of Po^, and Fq® on page 77 
after multiplication by appropriate powers of 10 to produce deci- 
mals of the same denomination as the other parts of the normal 
equations. (The factor 10“® applies to the whole of the “ 1 ” 
column.) 

The sums at the right of the normal equations are not the num- 
bers 0.879, 0.995, and 6.136 previously seen in the check under 
Table 3 but are these numbers to which have been added the corre- 
sponding entries of the “ 1 " column; the normal equations thus 
start off with a sum column that provides checks at the pivotal 
points of the solution (note the check marks in Rows 11, III, and 
IV). 

The solution proceeds according to the directions under “ How 
obtained.” The same system of solution has been seen in simple 
problems on pages 20, 33, and 66, and will be seen again on page 1B8 
and in Chapter XI. 

36, Conclusions from the solution of the normal equations. 

1°: From Row IV, 

S or E toF® = 0.042 ■ 10-" 

It follows from Eq. 21 on page 28 that 

<r^(ext) = 0.042 • 10"® ^5- (6 - 3) = 1.4 • lO"® 

Since this is only about one-third the prior tr* arbitrarily chosen: on 
page 76, we conclude that so far there is no indication of blunders in 
the observations or recording. 

2° : Rows 13, 12, and 11 in the solution on pages 82 and 83 jpve 
Xi = -0.308, X2 = 0.073, Xg = 0.071 • 10“* 

These used in Eq. 12, page 64, give 

Fp = i(XiFpi -1- XaFp^ + XgFp®) = -0.0000076 radian 
= —0.03 min. 
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COMBINUD BOLimON OF THE 

NORMAIi EQUATIONS, 

THE COMPUTATION 




UnhiownB 



Row 

10““Xi 

10-“X2 

10-’X3 

« 1 

I 

0.666 

0.346 

0.222 

-0.133X10“' 

2 


1.190 

-0.492 

-0.064 

3 



2.500 

0.073 

4 

' Faclora 



0 

6 

0.346/0.666 == 0.6106 

-0.211 

-0.136 

0.081 

n 

0.979 

-0.628 

0.027 

6 

0.222/0.666 0.3029 


-0.087 

0.062 

7 

0.628/0.979 =>0.6416 


-0.403 

0.017 

m 


2.010 

0.142 

8 

0.133/0.666 a 0.2364 



-0.031 

g 

0.027/0.979 » 0.0276 



-0.001 

10 

0.142/2.010 « 0.0706 



-0.010 

IV 



-0.042 


IS 



10-“Xi = ■ 

-0.3081 

12 



lO-'Xa = 

0.073>-X10-' 

11 



lO-'Xa = 

0.07lJ 

14 

0.182/0.666 

0.3221 



16 

0.071/0.970 

= 0.0726 



16 

0.474/2.010 

= 0.2368 



IV‘ 





17, 

0.222/0.666 

= 0.3929 



18 

0.628/0.979 

= 0.6416 



19 

2.010/2.010 

= 1 



IV* 





20 

0.667/0.666 

= 1.1623 



21 

0.663/0.979 

= 0.6772 



22 

0.689/2.010 

= 0.2930 



IV* 






(The powers of 10 written ot the tope of the " 1," (7^ C^, 
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OF 2wV“i AND 

THE 'WEIGETB OF THREE 

FUNCTIONS 


01 

0® 

0® 

Sum 


0.182X10“® 

0,222X10“® 

-0.667X10® 0.746V 


0.182 

-0.492 

0.262 

o.g4iv 


0.500 

2.500 

-0.094 

6.209V 


0.500 

2.600 

3.687 

6.673 

How obtained 

-0.111 

-0.136 

0.401 

-0.456 

I(-O.61O0) 

0.071 

-0.628 

0.663 

0.485V 

(2)+(6) 

-0.072 

-0.087 

0.258 

-0.293 

I(-0.3929) 

0.046 

-0.403 

0.425 

0.311 

n (+0.6416) 

0.474 

2.010 

0.589 

6.227V 

(3) + (6)+(7) 

0.043 

0.052 

-O.ISS 

0.176 

I(+0.2354) 

-0.002 

0.017 

-0.018 

-0.013 

II(-0.027e) 

-0.033 

-0.142 

-0.042 

-0.369 

lII(-0.0706) 

0.508 

2.427 

3.472 

6.307V 

(4) + (8)+(9) 





+ (10) 




Subst. from (11) & (12) 





into I 




Subst. from (11) 





into 11 

IlI-T-2.010 

-0.069 




I(-0.3221) 

-0.006 




II(-0.0726) 

-0.112 




III(-0.2368) 

0.324 




(4) +(14) +(16) 





+ (16) 


-0,087 



I(-0.3929) 


-0.403 



n(+0.6416) 


-2.010 



IIK-l) 


0.000 



(4) + (17) + (18) 





+ (19) 



-0.764 

i 

I(+1.1623) 



-0.449 


n(-0.6772) 



-0.173 


m(-0.2930) 



2.302 


(4)+(20) + (21) 





+ (22) 


and 0* columns are understood to apply all the way down.) 
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7<j = + "KsFq* = 0.0000582 radiau 

5= 0.20 min. 

7s = ')^iFn + Xa^’s* + XaFs® = 0.0000214 radian 
= 0.07 min. 

1 n® 

7j, = (XiPp^ + XaFp^ + XaFp*’) = 0.25 ft. 

in® 

7, = ^ (XiF,! + + MFa^) = -0.33 ft. 

in® 

7, = ^ (Xif?/ + Xafr® 4- Xa/i’r®) = 0.08 ft. 

u4.0 

for the six residuals. It is important to note that the numerical 
values of the derivatives required here are already worked out in 
Table 1, page 78. 

3° ; By using these residuals with Eq. 6, page 52, we find that the 
adjusted value of 

Angle P is 51“ 06'.25 + 0'.03 = 61“ 0G'.28 
Angle Q is 95“ 04'.5 - 0'.20 = 95“ 04'.30 
Angle R is 33“49'.5 - 0'.07 = 33“40'.43 
Side p is 1723.7 - 0.25 = 1723.45 ft. 

Side q is 2205.4 + 0.33 = 2205.73 “ 

Side r is 1232.7 - 0.08 = 1232.62 " 

Rmarh, Perfect dosure (third condition on p. 76) may be 
secured by lowering angle R by the trifling amount O'.Ol ; the value 
33° 49' ,42 BO obtained, along with the other adjusted angles and 
sides just written, will satisfy also the first and second conditions on 
page 76 to within 1 part in ^ million, which is about all we should oak 
for. Whenever, as happened here, one or more of the conditions 
failB owing to cumulated inexactness of rounding off, the computer 
is at liberty to manipulate the terminal figure of one or more of the 
residuals, raising or lowering it a unit or so to force the conditions. 

If not inconvenient, he will ordinarily (as was just done here) 
select, _the quantities of least weight for any such manipulations. 
The amount involved will be small compared with the standard 
errors of the final results (of, also p. 220), 
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4°: The weights and the standard erroK of the three Q functions 
(p. 77) are found as follows: 

From Row IVj the weight of the adjusted angle P ia 
1/0.324 . 10-“ ■ 10+“ = 1/0.324. In other words. oA is the 
variance coefficient of angle P. Then with = 4 23 • 10~® 
(p. 76), it turns out that the standard error of the adiusted 
angle P is (4.23 • 10 “ • 0.324)* = 1.2 ■ 10“+ radian = 0.40 min. 


So 


Anglo P = 51° 06', 3 zt 0'.4 


Prom Row IV® the weight of the adjusted sum of P + Q + 

1/0 or «, as predicted. Hence the sum of the adjusted andes 
would be written 


P Q + R = 180° absolutely 

From Row IV® the weight of the area ^pq sin P is 1/2.302 X 10^® 
Its si^andard error is therefore (4.23 X 10“® X 2.30 X 10^®)* =' 
312 square feet; therefore the adjusted value of 

The area is 1058028 ± 312 sq. ft. 

The area would better be written (105803 zfc 31) X 10 square 
feet, since not more than two figures of the standard error could be 
assumed known. In acres, 


The area = 24.2890 dz 0.0072 acres 

■nie area is found by using the adjusted values of p, g, and B and 
talcing Jpg sin R. Of course one could as well use Jgr sin P or Jpr 
sin Q for the ai'ea; one is as good os another. 

Exercise 1 . Prove by Eq. 9, page 40, that after adjustment the 
weight of the meBi is 3i little more thsiH double its weight before 
adjustment. 

Hint; By using Eq. 9, page 40, we find that 




= area*' 


[ 1 _j_ 1 cot®P 


= 1.12 X 10*® (1.37 + 1.07 + 2.24) 
= 6.25 X 10*® before adjustment 
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Therefore 

Wareo “ 0.19 'X 10“^* (before) 

We had 

'"area “ ‘Weight of G® = 0.43 X 10“^^ (after) 

The result stated follo'ws at once. 

Exercise g. (From L. D. Weld’s Tkeory of Errors and Least 
Squares, Macmillan, 1916.) Take the line AB, on which are lo- 
cated points C and D. The whole line and its segments are 
measiured with the same rule under similar conditions, the results 
bdng 

Xi = AC = 45.10 cm., mean of 2 observations 
X2 = AD= 77.96 “ “ “ 3 “ 

Xs= CD = 32.96 “ “ " 2 “ 

Xi^ OB = 98.36 « “ “ 3 » 

Xi = DB^ 66.55 « “ » 2 » 

Xi = AB = 143.65 " » " 4 " 

T I 1 I ■ — 

• COB 

Fig. 13. The line and ita eegments, corresponding to Exercise 2. 


Problem, Find the least squares values of the lengths. 
Take wi = 2, W 2 = 3, wg = 2, W 4 = 3, wg <=> 2, wg = 4. 

Conditions: 

= Xl + Xg Xg — Xg = 0 
F^ — X\ — Xg-^- Xg =0 

F® = aa ~ 3:4 + *6 = 0 
Show that the normal equations are as follows. 
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Solution: 

Ai = -0.1145, Aa = +0.0952, Xg = +0.1662 
Residuals: 

Vi = -0.0096 cm. 

Fa = -0.0317 “ 

Va = +0.0680 " 

Vi = -0.0517 “ 

Ffi = +0.0203 “ 

Fo = +0.0286 “ 

Adjusted values: 

AC = 45.110 cm. 

AD = 77.992 “ 

CD= 32.882 “ 

CB = 98.412 “ 

DB= 65.530 " 

AB = 143.522 " 

{AB actually turns out to be 143.521 cm., but the last decimal is 
raised one unit to satisfy the firat condition. The other two con- 
ditions are satisfied perfectly by the adjusted segments.) 

Exercise S. (a) By Eow IV in the solution of the normal equa- 
tions of the preceding exercise, the minimised value of 2 wV^ is 
0.0246. 

(6) Find 2)' wV^ by direct computations, using the values Fi, 
Y 2 , etc., found in the solution. Ans. 0.0246. 

Exercise 4 . Find the standard errors of AB and AD, taldng the 
standard error ff of a single measurement to be 0.05 cm. 

Exercise 6. (a) Show that the estimate of o- made from £ io7® 

is <r(ext) = 0.09. 


(by applying 
Eq. 12, p. 54) 
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(6) Show that, with (T = 0.05, x® = about 10, and ^(x®) = 0 02 
wherefore we might say that the discordance between the observed 
lengths of the segments is somewhat larger than one might expect 
from previous experience. 

Note: Since the individual mcflauremente wore not recorded 
there is no possibility of estimating a from the original observations’ 
i.e., we have no a-fint) to compare with the prior a and ir(exi). ' 

Exercise 6, The three inside edges of a rectangular parallele- 
piped are measui'ed with calipers and a linear scale; and the 
volume is measured in cubic units by filling it with meremy, which 
is afterward poured into a graduated cylinder. The results of a 
set of observations ai’e as follows: 




Mean 

71 

Standard 

deviation 

On edges paraUel to the x direction. 
On edges “ " " y direction. 
On edges “ " “ * direction, 
On the volume, 

A'l (cm.) 
Xi ■■ 

X, " 

Xi (oo.) 

711 

71.2 

713 

714 

Si (cm.) 
sa " 

83 “ 

S4 (co.) 


K randomness has been demonstrated, one may pool the 
st^ard deviations of the measurements on the three sides to 
pt on estimate of the standard error of a single observation on a 
Imear measurement. If <ri denotes the standard error of a single 
linear measurement, then one would write 


ffi® iest'd) = + risSa^ 

+ Kj + Tia — 3 


(Cf. Eq. 07 in Deming 
and Birge, oited on 
p. 29.) 


Jilt PT. ” 30), this estimate will be 

SSnf 1 ®^' standard error of a direct determi- 

nation of volume, one would likewise write 


vi* (est’d) = — 1^“° 

Wl — 1 

™h “aS be reliable 

enough. After obtaming estimates of ,ri and one would 
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assign weights to the observations Xi, Zj, Z3, and Z4 as follows 
(see Eqs. 18, p. 26) : 


nttr* 

Wi = j- 

njo-® 

Wi = — 5- 

Wi = — T 
(T4® 

(T*, as in Section 11, is an arbitrary factor of proportionality, the 
standard error of olDservationa of unit weight. If it is set equal 
to triS we should have the convenient system of weights, 

Wl, Wt, Ws, Wi = ni, 712, 713, — r 

The weights having been settled on, we can proceed. The 
one and only condition on the adjusted values is that 

whence wo put 

F = Xi — xix^t 

Suppose we need the standard error of the volume after 
adjustment; wo set 

G = Xi 

(a) Show that the one and only normal equation is LX = Fo. 
whence X = Fq/L, where 


L = Z4 


(b) In tabular form, the normal equation for finding X, S, and the 
weight of the adjusted volume, is as follows: 


■ 1 

, 1 . 

^ 1 H 



Xs’‘w3 ‘ Z 4 VI 


1 

2 

L 

Fo 

0 

I/Wi 

3 


— 0 /^ 


11 


—FgPo/Z/ 


4 




lU 



(1M)(1 - 1/Lv>i) 
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(c) The weight of the adjusted volume is ( 1 / 1 ^ 4 ) (1 - I/W 4 L). 

(d) (The standard error of the adjusted volume)® = a^[\/w^ 

(1 - IMi)- 

This standard error is smaller than tlio standard error of the volume 
before adjustment by the fractional amount l/Liui®. 


(e) The minimized sum of the weighted squares of the residuals, 
S, is J?'oX. 

(/) The estimate of ir® by external consistency (Sec. 13) is 


<r®(ea:<) 


Fok 
4- 1 


(g) What would you say if were much larger than your 
assumed value of e®, i.e., P(x) small? 

Svggutvm: Edges not parallel; lock of perpendicularity; measure- 
ments not so good as initially supposed (i.e., ai or 0-4 too small); 
just happened to be so. 


Qi) Show that after adjustment the standard error of the first 
is 

L 


/I 1-^ 
Wwi Xil 


36. Shorter method of computing the weights of a large number 
of functions.® The theory on which the weights of the three G 
functions were calculated in Sections 34 and 35 rests on the fact 
that® 


(wt. of G) 


. to J 




* To be omitted on first reading; the suggestion is that the reader return to 
tma after a study extending through Section 61 . 

® Gauss, Theoria Comhimlionia (cited in Sec. 13 ), Art. 29 . 



[Ch. VI] COMPUTATION FOR GEOMETRIC CONDITIONS 91 
where B', B", and B”' satisfy the equations 


LiiB' + + LisB"' = ^ I 

L 21 B' + L 22 B'' + LiiB'" = ^ 
BiiB^ + LiiB" + LiiB"' = 


(17) 


In other words, the aaxiliaiy constants B', B", and B'" will 
satisfy the normal equations (Eqs. 2, p. 73) if the “ C ” column 

[?] 

replaces the " 1 ” column. 

One may, if he chooses, solve for the Lagrange multipliers, and 
any set of auxiliaiy constants B', B" , B'" as well, by first of all cal- 
culating the reciprocal matrix 


A-i = 


Cll C12 Ci 3 
C2I C22 C23 
Cfli Caa C33 


(See Exs. 2, 4, and 
6 of Sec. 61.) 


(18) 


and then using it to calculate the Lagrange multipliers and the 
auxiliary multipliers in the manner following — 

Xi = + Fi^Cii -h Fq^ch 

X 2 = Eo^C 21 -f- Fo^Cii "h f^0^C23 * 

Xa =■ Fo^di -h Fq^h -f Fo^ca 


( 19 ) 
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B' = 

'F^(r\ 

. W - 

Cll + 

~F^G~ 
_ w _ 

Cia + 

~F^G~ 

. W . 

p 

Cl3 



cai 'h 

it 

C22 + 

it 

C 23 

B"' = 

it 

cai + 

t( 

C 32 + 

it 

C33 

t 


The Lagrange multipliers (X), after being calculated from 
Eqs. 19, are used in Eqs. 12, page 54, to compute the residuals Vt,' 
• • Vn, just as was done on page 81. The auxiliary constants B', 
B", and s'" from Eqs. 20 are used in Eq. 10 to find the weight of 
the function G. It will he noticed that the coefficients multiplying 
the c coeffidents in Eqs, 19 will already be available from the first 
step, outlined on page 70 and carried out numerically on page 77. 
The brackets in Eqs. 20 arise by cumulating squares and cross- 
products from Table 2 of the fourth step (pp. 72 and 79; summed 
numerically in Table 3 on p. 80). It is not difficult to extend 
Tables 2 and 3 to take account of a new G function any time it is 
desired to introduce one. 

The work then proceeds rapidly, the reciprocal matrix A"* 
being used over and over in Eqs. 20 for all the G functions. If one 
is working with a fairly good-sized number of G functions, this 
scheme will save considerable time over the direct computation 
illustrated in Section 34. 

A distinct advantage of using the auxiliary multipliers is that 
the redprocal matrix, once computed, is ready for use any time a 
new C column is produced, whereas, with the direct solution in 
Section 34 it is no little trouble to introduce a new C column after a 
solution has once been carried through. 

The three G functions used in Section 34 will serve for an illus- 
tration. To calculate the reciprocal matrix A“^ we take the coeffi- 
cients of the unknowns in the normal equations on pages 82 and 83, 
and put the umt matrix on the right of the equality sign, thus 
starting off with the equations 


0.565 ■ 10®® -f- 0.345 • 10®^ + 0.222 • 10®« = 1, 0, 0 ' 

0.345 ■ 10®® -1- 1.190 • lQ°y - 0.492 ■ 10®g = 0, 1, 0 » 

0.222 • 10®® - 0.492 • + 2.500 > 10®g = 0, 0, 1 . 


( 21 ) 
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The letters x, y, z designate the three unlcnowns that are to be 
solved for. Since there are three constant columns on the right, 
there will be three different solutions. The simplest way to 
obtain them would bo to follow the regular routine for solving 
normal equations, as illustrated in Sections 34 and 61. How- 
ever, one unfamiliar with that procedure may malte three sepa- 

1 

rate solutions. First, one would use the constant column 0. 

0 

By any method of solution whatever he would obtain 
a: = 2.458 X 10“ 

2/ = -0.876 X 10“ 
z 0.391 X 10“ 

0 

Second, he would use the constant column 1 and find 

0 

X = -0.874 X 10“ 
y = 1.226 X 10" 

z = 0.319 X 10“ 


0 

Third, he would use tho constant column 0 and find 

1 

X = -0.390 X 10“ 
y = 0.319 X 10“ 

z = 0.498 X 10“ 


The reciprocal matrix is simply a convenient way of filing these 
results systematically. It is written like this: 



2.458 ■ 10" - 0.874 • 10“ - 0.390 • 10“ 

-0.875 ■ 10“ 1.226 • 10“ 0.319 • 10“ 

-0,391 • 10“ 0.319 ■ 10“ 0.498 


( 22 ) 


The occasional failure of symmetry in the third decimal place 
comes from not carrying more figures; but what we have is good 
eno u gh. Supposing that the Lagrange multipliers have not been 
worked out, we should next compute them from Eqs. 19 as follows: 


Xi = -0.133-2.458-|-0.054-0.875- 0.073'0.391 = -0.309'| 
Xa = -1- “ 0.874- “ 1.226-h “ 0.319= 0.073 > 

10“X3 = -H “ 0.390- " 0.319-1- “ 0.498 = 0.071 J 


( 23 ) 
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(24) 


*“-■ [Sec. 36 ] 

These agree well enough with the values -0.308, 0.073, and 0 071 
already found in Section 35 (conclusion 2°, p. 81). 

The chief aim at present is to compute the auxiliary constants 
B'l B", B'" for each of the three G functions of Section 34. Gome 
back to Table 3 in Section 34 for the coefficients needed for Eos an 
we find that ‘ ’ 

F(rr(P- 

B' = 103 { + 0.182 . 2.468 - 0.182 ‘ 0.876 - 0.500 • 0 391 1 
= 0.0933 ■ 103 

B" = 103{ - 0.182 • 0.874 + 0.182 • 1.226 + 0.500 ■ 0 3191 
= 0.224 ■ 10® 

5"' *= - 0.182 . 0.390 + 0.182 • 0.319 + 0.500 • 0 498 

= 0.236 

These values used in Eq. 16 give 

— 0.500 - 0.182 . 0.0933 - 0.182 • 0.224 - 0.500 • 0.230 

= 0-324 (25) 

That is, the weight of = 1/0.324, in agreement with conclusion 
4” m Section 35, page 85. 

ForQ^ 

B = 10® j + 0.222 • 2.468 + 0.492 • 0.876 — 2.600 • 0 391 1 
= 0.00133 ■ 10® ’ 

B = lo^ [ _ 0.222 ■ 0.874 - 0.492 • 1.226 + 2.500 • 0 319 1 
= 0.00028-103 ' 

“ 1 ■ 0-300 ~ 0-492 • 0.319 + 2.500 • 0.498 

— l.UOlg 

These used in Eq. 16 give 
~ of G® = 2.600- 0.222 • 0.00133 -0.492 ■ 0.00028 - 2.500 ■ 1.0015 


(20) 


= -0.004 


(27) 


neSi4 ^ may suppose that thi 

gative result arises from not cariying enough figures. The low 
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est possible result, if all figures had been carried, rvould be 0 
Since m know what the result ought to be, we shall call it o’ 

whrrapon the weight of is infinity, as is already known (con- 
elusion 4“, Sec. 35, p. 85). ' 

ForG^ 

As sn exercise, the student should calculate B' B" and B'" fnr 
in like manner, obtaining ’ ’ 


5' = -1.808.10» 
B''= 0.865.10® 
B'" = 


(28) 


whereupon 


— of (t* = 10^® (3.687 +0.657 - 1.808+0.262 • 0.865-0,094 • 
= 2 . 300 . 10^2 

in agreement with conclusion 4“ in Section 35 (p. 85). 



(29) 


Remark. Tlio number of auxiliary constants B" B'" 
etc., in Eq. 16 is equal t'o the number v of conditions, i e the 
number of F functions. This is also the number of Lagrange 
multipliers (X), the number of equations in Eqs. 21, and the 
order of the reciprocal matrix. In contrast, the number of (? 
functions whose weights are wanted may be any whatever 
smaller or larger than the number of F functions. ’ 



CHAPTER Vn 


ADJUSTING SAMPLE FREQUENCIES TO 
EXPECTED MARGINAL TOTALS 

37. Statement of the problem. In social and economic surveys 
that are carried out by sampling, it is sometimes desirable to 
adjust the sample frequencies, or to adjust certain sample ratios, to 
TflalfR them agree •with certain corresponding totals or ratios that 
are known from other sources. Tliis happens, e.g., in the work of 
the Census: there is a complete count of certain characteristics 
for the individuals in the population, but in consideration of 
efficiency in time and costs, data on some characteristics are 
collected on a sample basis in the hmt place, and the tabulations of 
these sample data need to be adjusted to the complete count. 
Moreover, many of the cross-tabulations or joint distributions of 
population characteristics that have been obtained on a complete 
count are limited to a sample when the data are processed in the 
Washington office, and these croas-tabulations likewise need to be 
adjusted. The sample, except in extremely fine classifications, is 
entirely adequate for purposes of action (the only purpose of taking 
any survey in the first place) . The data of the sample are usually 
published as esiimaks of what would have been obtained by 
tabulating the characteristics for the entire population instead of 
only a sample thereof. This means that the sample is to be 
adjusted to certain totals that are known from other sources (as a 
complete count). 

The situation may be as shown in Fig. 14 m parallel tables for 
the universe and for the sample. For the universe, the marginal 
totals Ni, and Nj ore known from the complete count, but not the 
individual cell frequencies Ny; for the sample, however, tabular 
tion ^ves both the sample marginal totals Ui. and n.j, and the sam- 
ple cell frequencies rvy. After adjustment, the marginal totals of 
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the (adjuBted) sample and complete count will aKiee. The prob- 
lem is to write in the cell frequencies of the universe, with the. aiil 
of the sample, prcsciwing the marginal totals that are fix(,-(l by 
knowledge of the universe. It is tlus object of tliis chapter to show 
g phpmfis for performing such adjuatmontH. 


UNIVERSE SAMPLE 



Njj unknown njj known 

Marginal totals Nj and N Knawn Marginal totals n | and n | knowrt 
N known n known 


Pig. 14. Showing the sy.stein of milaliini fm- the ci ll fo tiiiciii'ifs .'itnl umr- 
ginsl totals of the univer.Hu und the H:ttii|>le hi (lie twir-diiiienviiiieil {)riihl(<tii. 


38. Cell frequencies and sampling errors, A .slati.stical table 
shows the frequences of oceurreiua; of the various i«(‘mlK!rs of 
subclasses within a population or uuivei-se, and i.s made uji of cells, 
one for each subclass. A two-diineiusimial universit is formed by 
the crossing of two classilicati()n.s, as depicted .stdiematicully in 
Fig. 14. An example is contained in Talih; 1, pag(( 107. The 
title of the table ordinarily dcscribeH the universe. The box lieud- 
ings over the columns define varioiiH mutually (,‘xelusive classes 
according to one system of classilicatiun, tmd the stub th aw lik(fwi.He 
for some othor system of elasaificalion. A memlKT of tin* miiversti 
will belong to one of the elas.se.s that are defined in the hetuUng, 
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and at the same time it will belong to one of the classes that are 
defined in the stub ; it is said to be a member of the subclass that ia 
defined by the combination or cross-elossilication of two particular 
classes described respectively in heading and stub. That is to 
say, a member of the universe must lie in one column or another, 
and at the same time it must lie in one row or another ; in the table 
it lies in the space common to a particular column and a particular 
row. This space is called a cdl. The number written in the cell 
is a ceUfraqumey and it shows how many members of this particular 
subclass were recorded in the enumeration of the univei’se, by 
sample or complete count. For instance, in Table 1, in the cdl 
deagnatedby the combined ages 14 and 15 (shown in the heading), 
for the state of New Hampshire (shown in the stub), is recorded 
a sample frequency of 395. When the tabulation is prepared by 
crossing three classifications, the result is a three-dimensional table. 
A three-dimensional table is usually printed as a sot of two- 
dimensional tables, rather than as a single table. These single 
two-dimensional tables all show tho aamo heading and stub, and 
each one represents the members of one class of the third classifi- 
cation, as the heading will sliow. A threo-diinonsional universe is 
depicted schematically in Fig. 15. Similarly, one may have foui'-, 
five-, or n-dimenaional tables. Tho sum obtained by adding the 
frequencies of an entire row or column is a marginal total or rim 
total, althou^ it could well be called a class total. 

When the data for the table arise from sampling, the frequencies 
(numbers) obtained are smaller than if the coverage had been com- 
plete, For instance, if tho sample is a so-called 5 percent sample, 
the ntunbera in the table will be only about 5 percent of what 
they would have been had a complete count been taken. It is 
not possible to perform the sampling in such a way that the sample 
frequencies are exac% 5 percent of what would be obtained on 
a complete coimt. If the sampling were so carried out, it would 
be sufficient merely to multiply every cell frequency by 20, and 
every marginal total also by 20, (The number 20 is spoken 
of as the sampling ratio, the reciprocal of 5 percent,) But be- 
cause of sampling errors, and possibly also because of certain biases 
that inevitably enter any survey, tho sample frequencies will 
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not be just l/20th of the frequencies that would be shown by a 
complete count. For the convenience of the user of a table, these 
sample frequencies are sometimes adjusted to some or all of the 



Fiq. 16. Showing the system of notation for the eell frequencies and mar- 
ginal totals in the three-dimensional sample. The cell shown shaded is desig- 
nated by the indices ijk. The sample fi'equency falling in this cell is »{,■{;. The 
corresponding adjusted deflated frequency is mijk, and the adjusted inflated 
frequency is Mijk. Some of the tube and slice totals are indicated. 

marginal totals that happen to be known from other sources, as 
by a complete count. This is a convenience to the user, because 
after adjustment the identical marginal totals are found m tables 
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having the same marginal specifications. Without adjustment, 
the frequencies might be alike enough for purposes of action, but 
perhaps not close enough tor identification. The known marginal 
totals that are used in on adjustment are spoken of as controls, 
or control totals. 

The adjustment is more than a conveuienco to the user; it 
HiminkViflH the Sampling variance to some extent; the more con- 
trols the smaller the sampling variance of the adjusted frequencies. 
(See the exercise at the end of Ch. V.) As a practical matter, 
however, this diminishing of the sampling variance should not be 
overemphasized, because biases and other difficulties may have a 
much greater effect than the sampling errors. 

In the work of the Census not all sample tables arc adjusted to 
all the known marginal totals. Even with the short cuts that will 
be described here, and which are more fully described elsewhere,'''® 
it may be more important to publislr the table at once, after merely 
multiplying the sample results by the sampling ratio, rather than 
to wait for adjustments to be made. One of the main advantages 
of sampling is quick processing, and this is particularly important 
for government planning in times of economic and social stress, in 
which the delay of only the brief time required for adjustment may 
not be advisable. 

39, Nature of the adjustment. It will perhaps be realized by 
now that the problems to bo taken up in this chapter are similar 
to the geometric ones in the last two chapters — similar in that 
' the conditions imposed on tho adjusted values are rigorous, not 
involving adjustable parameters. The same procedure for enforc- 
ing the least squares criterion will be found to give us an answer in 
this problem, as it did in the geometric problems. Here, however, 
short cuts will be described, which will greatly diminish the amount 
of computational labor and expense. 

® W. Edwards Demiog and Frederick F. Stephan, " On a least aquaies 
adjustment of a sample frequency table when the expected marginal totals 
ate known,” Annals qf MaOmnaiicdl Statiatica, vol. XI, No. 4, December 1040; 
pp. 427-444. 

* Frederick F. Stephan, “An iterative method of adjusting sample fre- 
quency tables when expected marginal totaJs are known,” Armala of M(Ako- 
matical Biatiatica, vol. XIII, No. 2, Juno 1942: pp. 166-178. 


k 
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40. A closer look at the problem. In estimating any cell fre- 
quency of the universe, such as Nij, three possibilities present them- 
selves: from the sample one may malce an estimate from the 
sampling ratio of the ith row alone, another from the aamplm g 
ratio of the jth column alone, and still another from the over-all 
sampling ratio N/n. Specifically, the three estimates would be 
nijNi./rii., riijN.i/n.j, and iiijN/n. These being simple multipli- 
cations of the observed cell frequency mj by three sampling ratios, 
viz., the sampling ratio Nt./ui. in the ith row, N.j/n.j in the jth 
column, and the over-all sampling ratio, iV/n. Because of Rampling 
errors, these three adjustments will not be identical except by 
accident, and though any of them by itself may be considered 
accurate enough, still, if the whole r X s table of universe cell fre- 
quencies were estimated by any one of these three adjustments, the 
marginal totals would not come out equal to the known values. 
This chapter presents three rapid methods of adjustment, which in 
effect combine all three of the estimates just mentioned, and at the 
same time enforce agreement with the marginal totals. These 
methods can be extended to varying degrees of cross-tabulation in 
three dimensions. 

Any method of adjustment must provide as its end product a set 
of adjusted frequencies that will satisfy the controls provided by 
the known marginal totals. In any problem of adjustment where 
the controls are intricate (many conditions), and where the adjust- 
ments are carried out by the hundreds and thousands, as they are 
in the Census, it is necessary to have a method that is straight- 
forward and self-checking; this is particularly important in three- 
way tabulations, where in one possible situation (Case VII in 
reference 1) the adjustment m one cell must be balanced by 
adjustments in at least seven others. It turns out, fortunately, 
that methods of the kind required in mass production can be 
devised (Secs. 45, 46, 48, and 49). 

41. The least squares requirement. By the method of least 
squares one would enforce the controls (conditions), and at the 
same time min i mi ze the sum 

Ui 


( 1 ^ 
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m stands for the cbsmed frequency in the ith cell, and im the ad- 
justed sample frequency therein. Ui is found in the sample survey, 
and mi arises in the adjustment. Here the denominator m is 
t ,p.lfftTi as the reciprocal of the weight of the ith cell. The bigger 
the frequency, the bigger the average sampling error (absolute 
error, not proportionate error), and accordingly the smaller the 
weight. It might be argued that the weight should be taken 
inversely proportional to tk,- rather than Wj, but, if the sampling is 
accurate enough for the purpose intended, it will make little 
difference which is used. Strictly, in random sampling, the 
redprocal of the weight of tin is ripngn, which is nearly equal to 
riijqij, where p and q have them usual connotations. But since 
factors proportional to the weights may be substituted for them, 
it is sufficient to use tin as the reciprocal of the weight in cell ij, 
since the values of gy do not usually vary much over the table. 
In stratified sampling, the weights are still closely inversely pro- 
portional to rii], 

42. The two-dimensional problem. Suppose that the data on 
two characteristics (e.g., age and highest grade of school com- 
pleted) are obtained for each member of a universe of N individu- 
als, but that tabulations of the complete data provide either 

Case I. Only one set of mai’giual totals, Ni., iVg., • • • , Nr.', 


or 

Case II. Both sets of marginal totals, viz., Ni., N 2 ., • • • , Nr., 
and N.i, N. 2 , ”• ,N.t (See Fig. 14.) 

The nature of the tabulations is presumed such that it is not 
feasible (too expensive) to count the numbers iVy ia the cells, as 
would be done if one character were crossed with the other in 
tabulation. Suppose, however, that in a sample of n individuals 
selected in a random manner from the universe, the two characters 
are crossed with each other, so that not only all the s -t- r marginal 
totals n.i, • • • jtir.oi the sample are known, but also every one of 
the numbersn,v(i = 1, 2, • • ■ , r; j = 1, 2, • ■ ■ , s). The problem 
is to estimate the unknown frequencies Nij in the cells of the uni- 
verse. This will be done by first finding the calculated or adjusted 
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sample frequeacies niij and then inflating them by the inverse 
sampling ratio N/n. 

For the least squares solution we seek those values of wiy that 
minimize^ 

S = n<y)2 (2) 

wherein the m,-,- are subjected to conditions of Case I or Case II. 

Case I: One set of marginal totals knovm. Assume iVi., N^., • • • , 
Nt. to be known. Then we require the marginal adjustments 

E mi = m. t = 1, 2, • • • , r (3) 

3 

These r equations constitute r conditions on the adjusted jwij', 
corresponding to Eqs. 3 of Chapter IV, page 50. Assuming that 
the adjusted values of the mij have been found, let each take on a 
small variation Smj-j ; then the differentials of Eqs. 2 and 3 show that 



(one equation) 

(4) 

Tlij 

£ hifUij = 0 i = 1, 2, • • • , r 

(r equations) 

(5) 


i 


Multiply now Eq. 5i by the arbitrary Lagi’ange multiplier - X(, 
and add Eqs. 4 and 5 to obtain 


m/ - riii 




— = 0 (one equation) 


,( 6 ) 


By the same argument that was advanced in Section 27, page 54, 
one may now set each brace equal to zero. The r Lagrange mul- 
tipliers are then no longer arbitrary, but each must satisfy the 
resulting relation 

mj = «ij(l + ^i) (7) 

® The sign ^ will denote sununation over all possible cells, unless otherwise 
noted. 2 will denote summation over all values of i, and similarly for an 
i 

inferior j or 7c. The dot in n,j will signify the result of summing the mi over 
all values of % in the jth column. 
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The adjusted frequencies mij can be computed at once as soon as 
the Xi axe found. To evaluate them one may rewrite the oon- 
ditiona (3) using the right-hand member of Eq. 7 for my, obtaining 

Wi. = + Xf) (8) 

way to axrive tit this same relation is to sum each member 
of Eq. 7 in the ith row. However obtained, X,- is now known, since 
mi, and Ui, are known, and in fact Eq. 7 now reduces to 

Wli. 

= (9) 

The adjustment is thus a simple proportionate one by rows, the 
cells in any one row all being raised or lowered by the proportion- 
ate adjustment in the row total. Case I thus amounts to r inde- 
pendent one-dimensional proportionate adjustments, one for each 
row; and any one or all may be earned out, as desired. This 

result can be obtained by a simpler approach but is presented in 

this way for consistency with later cases. 

The minimized sum of squar® may be computed directly, or 
from the row totals by seeing that 

5 = £ — (m,-. — n;.)® (10) 

, »■ 

The term (wij. - Ui.Y/ni. for the ith row may be considered sepa- 
rately, and used as 'with s — 1 degrees of freedom, or all rows 
may be combined into the minimized S as given in Eq. 10, and used 
as X® with r (s — 1) degrees of freedom. 

Case II; Both sete of marginal totals knoion. Here the adjusted 
cell frequencies must satisfy not only conditions (3) but also 

L mij = m.,- j = 1, 2, • • • , s - 1 (11) 

t 

there being now a total of r -]- s — 1 conditions. In both cases, 
mi. (12) 


( 13 ) 
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la other words, Jra,-. and m.y are the deflated marg in n.] totals, i.e., 
Ni. and N.j divided by the actual sampling ratio N/n. The m,-. 
and are not independent, because 

N.i + N.2 + ■ • • + N.a = Ni. + Ni. + ■; ■ + Nt. = N (14) 

It is for this reason that if i runs through all r values in Eq. 3, 
then j can run through only s — 1 in Eq. 11. A similar equation 
also exists for the marginal totals of the sample, namely, 

w.i + ^1.2 + • • ■ + = ’ll. + na. -|- • • • + n,.. = w ( 16 ) 

Solution of the iwo-dimensional Case 11. In addition to Eq. 6 
we now have also 

2 Snij = 0 j = 1, 2, • * •, s “ 1 (16) 

i 

which comes by differentiating Eqs. 11. By addition of Eqs. 4, 6, 
and 16, after multiplying Eq. Si by — Xi and Eq. 16j by —/ij, we 
obtain 

(17) 

fl'i} 

Equating each brace to zero, as before, we find that 

mij = n,y(l + X< + fij) (18) 

wherein /xj is to be counted 0. The adjustment is now no longer 
proportionate by rows, but involves every cell. 

To evaluate the Lagrange multipliers in Eq. 18 we may sum the 
two members downward and across in Fig. 14 and obtain the 
r + s — 1 normal equations 

rii.Xi + 2 nijnj = m. — ni. i = 1, 2, ■ • r 

3 

2 + n.jiij = m.j - n.j j = 1, 2, ■ • s - 1 

i 

These can be reduced for numerical computation. The top 
row solved for X,- gives 

Xi = — {mf. - 2 nauj] - 1 (20) 

^t. 3 

whereupon by substitution into the bottom row of Eqs. 19 we 
arrive at the s — 1 normal equations. 
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Ml Ma 

• • • M»-l 

= 1 

“ T Wi. T ni. 


i nj, 

i rii. 

x-i 

n.8 “ V „ 

t »i. 

» »ii. 

m.2 — 2* 

i Wj, 


■ 

(21) 


n.*-i i, 

t 

i fti. 



0 


Because of symmetry in the coefficients, those below the diagonal 
are not shown, indeed, in the systematic computation already 
shown in Section 33 (p. 73), they are not used. The 0 in the 
bottom row is appended for the computation of the minimized S, 
if desired. The number of Lagrange multipliers to be solved for 
directly is s — 1, and the remaining ones come by substitution 
into Eq. 20, jU, being counted 0. 

A simple procedure for calculating the coefficients in the normal 
equations (21) is to set up a preparatory table by dividing each 
Tii,- in the <th row by 8'lso to write down rtnj\/ni. for that 
row, for use on the right-hand side of the normal equations (com- 
pare Tables 1 and .2). In machine calculation the constant 
divisor Vjh. would be left on the keyboard until the entire ith row 
is divided; or, if reciprocal multiplication is preferred, the multi- 
plier Ify/ui, would be left on the keyboard. From this preparar 
tory table, the cumulation of squares and cross-products in the 
vertical gives the required summations for the coefficients. The 
sum check would be applied in the usual manner. 

43. A numerical example of the two-dimensional Case n. The 
fact is that in practice one need not bother about forming and solv- 
ing the normal equations because they will be displaced by a 
simplifying iterative procedure, to be explained in a later section. 
For illustration, however, we may do an example both ways, first 
using the normal equations and the adjustment (18), later on 
accomplishing the same results by the quidcer method. 

We may start with the unitalioized numbers in the 4X6 array 
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of Table 1, assuming these to be the sampling frequencies w,-,- to be 
adjusted. Actually, they were obtained by deflating l/20th (for 
a supposed 6 percent sample) the New England age X state table 
on p. 1108 of vol. 2 of the Fifteenth Census of the U. S., 1930, then 
vaiying the deflated values by chance with Tippett’s numbers to 

TABLE 1 

A TASnS OF SAMFL19 FnliqtnilNCIEB, A 5 FBRCBNT SAMFUB OF NAUVE WHCm 
fbhbonb op natiyb white pahbntaqe attending school, bt age by state: 
New England, 1930 


{The cdjusted freguency mij in each cell is ahovm itoMdged just heUno 
the corresponding sample frequency nij) 


Age 

7tol3 

14&15 

16&17 

IS to 20 




j = 

1 

2 

3 

4 

ni. 



H = 

0.0118 

0.0149 

0.0012 

0 

mi. 

State 

i 

Xi 






Maine 

1 

-0.0146 

3623 

781 

687 

313 

mi 




seis 

781 

550 

308 

5353 

New Hampshire 

2 

-0.0003 

1670 

396 

261 

Wm 

2371 




1588 

Jfll 

m 

WM 

3395 

Vermont 

3 

0.0234 

lil 

El 

264 

11,6 

2352 




tm 

WSm 

m 

119 

3433 

Massadhusetts 

4 

-0.0162 

10638 

2465 

1706 

1100 

16869 




lom 


1680 

IW 

15786 

Rhode Island 

6 

-0.0230 

im 

363 

171 

mm 

2369 




la 

S60 

167 

mm 

8330 

Connecticut 

6 

-0.0034 

3882 

Wm 

644 

339 

6622 




S91B 

M 

BiS 

338 

5683 



n.i 

"mm 

6260 

3493 

2237 

33837 




egsrr 

sm 

SJIBS 

3313 

33837 


The adjusted frequencies mu (italicized) are rounded off, hence when 
Bumzned may occasionally disagree a unit or so with the expected marginal 
totals (also italicized). The latter arise by deflation from the universe rather 
than by direct addition of the mif. and ft,- are found in the solution of 
Eqa. 20 and 21. 
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get fictitious sampling frequencies ny. The italicized entries in 
Table 1 represent the final (adjusted) frequencies triij, and it is 
these that we now set out to get. Wc start off with the sampling 
frequencies iHj and the known marginal totals m.i, m. 2 , etc., where 
17 H. = Ni.n/N, m.j = N.jn/N, as in Eqs. 12 and 13. The 
Lagrange multipliers shown along the left-hand and top borders 
arise in the calculations now to be undertaken. 

TABLE 2 

Each BAUPUII FEtEQUENCT IN TAHLB 1 DIVIDBC BT THE CORHEBFOmilN'a VRi, 


. This operaiion would ordinarily be done a row at a time. 



J = 


Sum 

1 

2 ' 

3 

4 

i ■= 1 !■' 

40.80 

10.76 

7.67 

4.31 

72.32 

144.04 

2 

32.24 

8.11 

6.15 

3.18 

49.19 

97.87 

3 

32.02 

8.04 

5.44 

2.30 

60.16 

98.64 

4 

83.08 

10.40 

13.56 

9.21 

125.19 

251.12 

6 

34.01 

7.27 

3.62 

3.17 

47.97 

90.54 

6 

61.77 

11.43 

7.26 

4.62 

76.61 

160.49 

Sum 

284.21 

66.09 

42.69 

26.78 

420.33 

839.60 


Table 2 is the preparatory table, advised at the close of the last 
section. It is derived from Table 1 by dividing the ith row of 
sample frequencies by \/ni.. For example, the entry 8.64 in the 
cell i = 3, j = 2 comes by dividing, 419 by V2352, 419 being the 
entry in the cell of the same indices in Table 1, and 2352 being the 
sum of the third row . The sums at the bottom and right-hand side 
are for cheeking the formation of the normal equations. The 
cumulations of squares and cross-products along the vertical give 
the summations required for the normal equations (Eqs. 21), 
which now appear numerically As Eqs. 22. 
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Row 

Ml 

Ml 

M3 = 

1 

I 

7413 

-3640 

-2364 

3197 X 10“* 

2 


4441 

-644 

2366 (22) 

3 



3129 

-3222 

4 




0 


Performing the solution by any favorite procedure one will obtain 
M = 0.01182 M 2 = 0.01490 MS = 0.00119 (23) 


whereupon by substitution into Eq. 20 comes 


Xi = -0.0146 
Xa = -0.0003 
X 3 — "1-0.0234 


X 4 = -0.0162 
Xs = -0.0230 
Xe = -0.0034 


(24) 


The next step is to compute the Tny by Eq. 18, Table 1 is now 
bordered with the Lagrange multipliers for a convenient arrange- 
ment of the factors required, and the calculation is completed. It 
will be noted that, for example, 

maa = 419(1 -f- 0.0234 -|- 0.0149) = 435 (25) 

The ?nij thus calculated are shown italicized in Table 1. The 
Tng.rginB.1 totals, found by adding the just calculated, do not 
agree exactly everywhere with the expected totals, because of 
rounding off to integers: the errors of closure, however, are alight, 
and it is a simple matter to raise or lower some of the larger cells 
by a unit or two to force exact satisfaction of the conditions, if this 
is desired. (Compare with the triangle problem on p. 84.) 

44. The three-dimensional problem. Here the AT cards of the 
universe are sorted and coimted for one and perhaps a second and 
third characteristic, and possibly crossed by pairs in various 
combinations (Cases I-VII). The sample of n, however, is crossed 
by all three characteristics, which is to say that the cell frequencies 
niji, are all loiown (refer to Fig. 15), As before, the adjusted fre- 
quencies are required, 
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Case I: One set of slice totals Jmoion. Assume the slice totals 
Ni.., Ni..,---, Nr., to be known; the conditions are then 


TIT ^ 

£ miih = mi,. = Ni.. —> 

ik N 


i = 1, 2, 


(26) 


(27) 


being r in number. The summation to be minimized here is 

ntjk 

being similar to that in Eq. 2, except that now there are three 
indices to be summed over instead of two. Following a procedure 
similar to that used before, we differentiate Eqs. 26 and 27 and 
introduce the r Lagrange multipliere X,-.. with Eq. 26. The steps 
are id^tical with those of the two-dimensional Case I and the 
result is at once ’ 


m^jh — nijhO- + Xi..) = nijh-^ (281 

' ' 

This adjratment, like that shown by Eq. 9, is a simple proportion- 
ate one, but this time by slices rather than by columns. All cell 
frequencies having the same i index are raised or lowered in the 
same proportion. 

Case 11: Two sets of slice totds known. Here, in addition to the 
shoe totals of Case I we know also 

whence arise the 5 — 1 additional conditions 

5 J = 1, 2, ■ ■ s - 1 (29) 

Using the La^^ge multiphers X.,-. here, and X,-.. with Eq. 26 as 
before, we find that ^ 


^Hk = + Xi.. + X.y.) 


(.ou; 


atetotb”.? ’’',^*"1 “‘ju'tosat is proportm- 

and L ^iik/riijk being constant along the ijth tube 

m fact equal to mdepeutot of k UufortaaMy 
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we do not here Icnow the face totals nuj. and are unable to make use 
of the proportionality as we shall in Case IV. 

To solve for the r + s — 1 Lagrange multipliers we sum the 
members of Eq. 30 over j and then over i and arrive at the nnrmii.l 
equations 


i 

E + n.jXj. = m.,-. - n.j., j = l, 2, ■ ■ s - 1 


(31) 


These can be reduced to s — 1 equations in precisely the Hn TWft way 
that Eqs. 19 were reduced, but, because of the great advantage of 
the iterative process to come further on, we shall not pursue the 
reduction here. 


Case III: AU three sets of slice totals known. All slice totals 


N.i„N.2.r'; N... 

iV..i, N..2, • • •, N..t 

now being known, in addition to conditions (26) and (29) we 
require here 


E — in.,]c — N,,]e J.J., k — 1, 2, • • 4 1 (32) 

which makes a total of r + (s — 1) + (i — 1) or r + s + < — 2 
conditions. The same kind of manipulation as used heretofore 
gives 

Wlij* = nijkO- + Xi.. + X.y. + X..fc) (33) 

with X.s. and X..< to be counted zero. The adjustment is no lonpr 
proportionate by slices or tubes, but involves every cell. In 
practice, once the normal equations are solved and the Lagrange 
multipliers worked out, one proceeds very much as in the two- 
dimensional Case II: for each of the t slices, corresponding to the 
t values of k, there will be a two-dimensional adjustment, the 1 in 
Eq. 18 being replaced now by 1 -|- X..ii:. 

The normal equations for the Lagrange multipliers can be found 
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by performing double summations on Eq. 33. The result is 
fii.lKi.. + 2 = mi.. — m.. 

3 k 

Z w, 7 .X{.. + n.jXj. + Z n.jk>^.,k = m.j. — n.j. 

j=l,2,-;s-l 

Z Wi.iXi-,. + £ w.jfcX.j. + n..kK..k = m..k — n..k 

A = 1, 2, 1 

If these calculations were to be carried out, one would simplify the 
computation by solving the top row for X,-.., getting 

~ ^ 2 - 1 (35) 

}■ i; ' ' 

and then substituting this into the middle and last rows of Eqs. 34 
to get a reduced set of s + < — 2 normal equations for the T-ngr o n g f , 
multipliers \j. and \..k, the numerical values of which when set 
back into Eq. 35 give the Xj... In all the summations of Eqs. 34 
and 35, X.,. and X,,{ would be counted zero. But here again, the 
iterative process to be explained later will displace the use of normal 
equations, so actually we are not interested in reducing them. 

CaselV: One set of face totals known. It may be that the rs face 
totals 

Nil., Ni2., • • •, Nij., • • •, Nra. 

are knoim from crossmg the i and j characters in the universe. 
The conditions are then 


? (34) 


Z mijh = mii. = Nij. 

h 


n 

n’ 


i = 1, 2, 
3 = 1, 2, 


The adjustment here turns out to be 


r 

8 


(36) 


= nijk{l + Xy.) (37) 

but by summing both sides over the index k to evaluate X.-,- it is 
seen that 

Wij. = -}• Xij',) 


(38) 
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whence 

mil. 

nUik ~ l^ijk (39) 

This adjustment is thus proportionate by tubes, like that in Eq. 30, 
though here the factor mij./riij. is known and Eq. 39 can be applied 
at once. 

Case V: One set of face totals, and one set of slice totals known. 
Sometimes, in addition to the rs face totals of Case IV, the slice 
totals 

wUl also be known, in which circumstances the conditions (36) are 
to be accompanied by 

£ fOiijk — = N ..k'T.* k = 1, 2, • • •, i 1 (40) 

a Jy 

The same procedure as previously applied yields now 

mijk = w<jfc(l + Xy. + X,,*) (41) 

with X..t to be counted zero. Summations performed over k, and 
then over i and j together, give the normal equations 

n'ij.Xy, "I” ^^ni}kf^,.k ~ wiy. nfj, 

k 

£«iifcXy. + n..s,X..fe = m..k — n..i 
ii 

The number of equations is rs + < — 1, since X..t does not exist. 
As before, a simplification can be effected by solving the top row 
for Xy, and maidng a substitution into the lower one, but, because 
of the great advantage of the iterative process to be seen further 
on, we shall not carry out the reduction. 

Before going on it might be noted that although this case is thre^ 
dimensional, it reduces to the two-dimensional Case II if one 
considers that ij. is one index running through the values 11, 12, 
• • ', 21, 22, • • rs, and that .,k is a second index running 
through the values 1, 2, • • •, t. This can be seen by the smularity 
between Eqs. 42 and 19. 
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Case VI: Two sets of face totals known. If in addition tn 
face totals of Case IV, the face totals * 

N.12, • • •, N.,i 

are also known from further crossing the j and k characters in thp 
universe, we shall require 

= = / 43 V 

• N A = 1, 2, ■ . i - 1 J 

m addition to the conditions (36). In place of Eq. 39 of Case IV 
we now find that 

mijh = n,jji(l + X<y. + X.yj) ^44j 

m which \jt is to be counted zero for all j. No simple relation 
Pid. ^ a, 39 i. P«ible h™, L to « 

prqurtomte by tubes, ■ tin Lagrange muWpIiora must be evalu- 

Pn li summing the members of 

Eq. 44 over ft and z in turn, resultmg in the normal equations 


+ £ nijkX.jic = m,-y, - Uij. 

L nijkhi. + == m.jk — n.jk 


(46) 

3m« X.,, ta net for any v^„e, the „ Jber of eqnatioaa 

f I ^ = «(»• + « - 1). They break up at once into 

fact thttobl " ' “i. ^ equations, one set for every j value. In 
fact the problem can be considered as s sets of the two-dimensional 

™n L f.,ltl7 Z be looked 

Each specifications of the two-dimensional Case II. 

“eo bi the same manned 

tnat Eqs. 19 were reduced. 

AUtotabuowbeing 


Zmijk = mii. =Ni,.-, 
Xmijk = =JV.yj,^f 

'Lmijk^rrn.k =Ni.k^, 

’ N 


i = 1, 2, . 

• ■, r 


II 

• ■> s 


J = 1, 2, . . 

•1 s 


ft = 1, 2, • ■ 


1 

f = 1, 2, . . 

•,r- 

1 

ft = 1, 2, . . 

t- 

1 


(36) 

(43) 

(46) 
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The adjuBting relation is 

niijk = nijkO- + Xjj. + \.jk + Xf.fc) (47) 

in which \.jt is to be counted zero for any j, for any k, and 
X,’,{ for any i. The normal equations for the La grange multipliers 
are 

nijlSij. + 22 WyfcX.jj, + 22 nijk\i.k = my. — ny. 

k h 

2 "I" ^.jk^.jk "b ~ m.yj — n.jk 

i i 

£ Wij-jiXij. + 22 ni3k\jk + ni.k'Ki.k = rrii.k — ni.k 
i i 

being rs + ri + si — r — s — i+1 in number. They can be 
reduced in the same way that previous normal equations have 
been reduced; but here again, the iterative process will render the 
use of normal equations unnecessary, except for theoretical pur- 
poses, e.g., justification of the iterative process. 

45. A simplified procedure — iterative proportions. The num- 
ber of Lagrange multipliers in any problem is equal to the number 
of conditions imposed on the adjustment (Sec. 27). Here the 
conditions have appeared in sets, depending on which marginal 
totals are involved. By a comparison of Eqs. 9 and 28 on the one 
hand, with Eqs. 18, 30, 33, 41, and 47 on the other, we see that 
wherever there was nnly one set of marginal totals involved we 
came out with a simple proportionate adjustment, but that in all 
other cases it was not so; the Lagrange multipliers involved were 
unfortunately related to one another through normal equations. 

We need a simplification. It is a fact that as a first approxima- 
tion the adjustments may all be considered proportionate, in 
either the horizontal or the vertical. We shall be able to write 
down an expression for the error in this approximation, and shall 
be able to reduce it sufliciently by a succession of proportionate 
adjustments. 

Take the two-dimensional Case II for an example. In Eq. 20 

one may recognize (l/wi.) £ wyjW 0^ 0 weighted average of /ly for 

3 

the ith row. There will he a weighted average of jity for the first 
row, another for the second, etc., one for each value of z; conse- 
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quently one may appropriately speak of the ith average of nj, 
writing it f-av nj. Substituting from Eq. 20 into 18 one then sees 
the adjustment (18) appear as 

\ 

— + W - t-av j (49) 

If, on the other hand, /ij had been eliminated from Eqs. 19, instead 
of X<, the result would have been 

mu = ny + \i- i-av (50) 

From either Eq, 49 or 50 it is clear why the adjustment (18) is not 
proportionate by rows or columns, and why Case II does not break 
up into r or s seta of Case I; the reason is that Hj in any cell is not 
necessarily equal to the average lij for that row, nor is Xi in any cell 
necessarily equal to the average Xifor that column. If nevertheless 
one were to make the simple proportionate adjustment 

m,7 — fiij (51) 

“i. 

along the horizontal in the fth row, the horizontal conditions (3) 
will be enforced but not the vertical ones (11) ; i.e., it will be found 
that nii,' = mi., but that usually not all m./ = m.j. This is 
because Eq. 61 effects only a partial adjustment, each mi/ being 
in error through the disparity between the nj proper to the jth 
column, and the average of all the tij for the itti row, as seen in 
Eq. 49. This error can then be diminished by turning the process 
around and subjecting these mi/ to a proportionate adjustment in 
the vertical according to the equation 


niij — niij- 

m.j 


(52) 


which may be considered an application of Eq. 50 wherein the dia- 
, parity between any X* and the average Xj for the jth column has 
been neglected. It is the vertical conditions that will now be 
found satished, but perhaps not all of the horizontal ones, because 
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some of the row totals may have been disturbed. The cycle 
initiated by Eq, 51 is therefore repeated, and the process is con- 
tinued until the table reproduces itself and becomes rigid with the 
satisfaction of all the conditions, both horizontal and vertical. 
The final results theoretically do not coincide with the least squares 
solution, but in practice they usually do, closely enough. 

Usually two cycles suffice. In practice the work proceeds 
rapidly, requiring only about one-seventh as much time as setting 
up the normal equations and solving them. The Tables 3-6 show 
the various stages of the work when the method of iterative pro- 
portions is applied to the sample frequencies of Table 1. It will 
be noticed that the results of the third approximation (Table 5) 
are final, since if the process were continued, the table would only 
reproduce itself. 


TABLE 3 

Tan UBTBOD or irEnATZVii fhopobtionb APFunn to xhh data or tablb 1 

(PIHBT BTAQHl) 

A jrroporlionaie adjuslment hyrows, by Eq. 61, Note that lai ■» m,., 
hat that m,/ ^ ni.j. ' 



3 = 1 

2 

3 

4 


OT{, 

i = 1 

3608 

778 

666 

312 

6263 

6262 

2 

1686 

309 

264 

167 

2396 

2396 

3 

1606 

433 

273 

120 

2432 

2432 

4 

10476 

2441 

1698 

1163 

16766 

16766 

6 

1660 

349 

169 

162 

2330 

2330 

6 

3910 

863 

648 

841 

6662 

6662 


22846 

5263 

3495 

2236 

33839 


Wi./ 

22877 

6285 

3482 

2213 


33837 


46. Iterative proportions in three dimensions. The same 
process can be extended to three or more dimensions with an even 
greater relative saving in time. To see how the method of itera- 
tive proportions applies in one of the three-dimensional cases, we 
may go back to the three-dimensional Case III. By the substi- 
tution afforded throu^ Eq. 35 the adjusting Eq. 33 may be put 
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TABLE 4 


A CONTINTfAOION OF THE PHOCEBS INITIATED IN TABLE 3 

(second stage) 


Tkeji^eam TcMe 8 are now ^justed propoHionalely by columns acmdina to 
Eq. 68. The verHeal totals m.] and m.j now are equal, biU the agreement of the 
horigontdl totals accomplished in Table S has been slightly disturbed. 



TABLE 6 


The ctclb commenced again 
(third stage) 


Z i i ^ o^ustment by rows, accord- 

tJ and vertical conditions satisfied, they are cZ 

mdered find. The agreement vnOi the my in Table 1 should b7noted 
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into the form 



/mi.. 

mijk = Tiijk [ h \j. + X..H, — f-av X.,-. 

\ni.. 

- f-av X..i.^ 

(53) 

Equally well it could have been written 



fm.j. 

mijk = Uijk 1 h Xi.. -t- X..i — j-av Xi.. 

\n.i. 

- i-av X..fc^ 

(54) 

or 

V 


m,ijk riijk ( -f- X,-.. -t- X.y. A:-av Xy., 

\n..k 

, ~ fc-av X.y. 1 

(55) 


Any of these three equations shows why the adjustment (33) is not 
proportional by slices, and why this case does not break up into r 
or a or i sets of the three dimensional Case I. As a first approxi- 
mation it does, as is now clear from these three equations, and by 
malting successive proportionate adjustments we may thus arrive 
at the final values. To go about the work one could first calculate 


the values of 



then 

/ mi.. 

mijk =nijk — 

n<.. 

(56) 

followed by 

ir 1 m.j, 

mijk -mijk' , 

m.j. 

(57) 


ut n m.,k 

mijk — mijk If 

(58) 


These three successive adjustments would constitute a cycle, 
which would then be repeated in whole or in part until the table 
becomes rigid with the satisfaction of all three sets of conditions. 

47. Simplification when only one cell requires adjustment. On 
occasions it happens in sampling that one is especially interested 
in one particular ceU of the universe, and would like to have a 
result for it in advance before the other cells are adjusted. Some- 
times it even happens that the others individually are of no particu- 
lar concern. In such circumstances one merely places the cell of 
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interest in one corner of the table by an appropriate interchange of 
rows and columns, and then compresses the rest of the table into 
the adjacent to it. In the two-dimensional Case II one would 
thus work with a 2 X 2 table, one corner cell being the one of special 
interest, the other three being the result of compression. The 
TnaTEiTifll totals of the row and column belonging to the cell of 
interest are unaffected. For illustration we may suppose that 
from the sample shown in Table 1 we require only wioi. We then 
start with the 2 X 2 Table 0, which is derived from Table 1 by 
compression. Commencing with Table 6, one might first adjust 
by rows according to Eq. 51, then by columns by Eq. 52. One 
cycle of iterative proportions is sufficient, as is seen in Table 7, 
and the value 3915 found for mei is in good agi’eement with its 
value shown in Tables 1 and 5. The scheme of compression 
provides a quick method of getting out an advance adjustment for 
a cell of special interest, and the result so obtained will ordinarily 
be in good agreement with what comes later when and if all the 
cells are adjusted. 

In the three-dimensional Cases II, III, V, VI, and VII, one 
compresses the original table to a 2 X 2 X 2 table, and then uses 
the method of iterative proportions. (The other cases do not 
require consideration, since they are proportionate adjustments 
wherein one is already at liberty to adjust as few or as many cells 
as he likes without altering the equations or the routine.) The 


TABLE 6 

DeSIVBII ITROU TiniiB 1 BY COUPBCSBION, THB OSILIi i => Q,j = I, KDqttlBINa 

ADJUSTMENT 



j = 1 

i =2-4 

ni. 

mi. 

i = 1-6 

18966 

9260 

28216 

28176 

1 = 6 

3882 

1740 

5622 

6662 


22847 

10990 

33837 



22877 

10960 


33837^ 
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aame procedure can be extended to the adjustment of two cells, the 
only modification being that in two dimensions we rLbH compress 
toa2X3ora3X3 table, depending on whether the two pp.llp 
do or do not lie in the same row or column. In three 
we compress toa2X2X3, ora2x3X3, ora3X3X3 
table; the first if the two cells lie in the same i, j, or k tube, the 
second if they lie in the same slice but not in the same tube, the 
third if they are in separate slic^. 

TABLE 7 

A PBOPOHTIONATB ADJUSTMUNT OJ TABIB 6 


Bows adjusted by Eq. 51 Colunms adjusted by Eq. 62 


18938 

9237 

28167 

18962 

9213 


3910 

1762 

6602 

3915 

1747 

6662 

22848 

10989 

33837 

22877 

10960 

33837 


Conclusion: msi ■* 3915 


48. The Stephan method. An iterative procedure devised by 
Stephan^ has the advantage of being in theoretical agreement with 
the least squares solution. It is moreover self-checking and self- 
correcting, and requires the writing of only a few figures. Only 
one table is required, since the factors required in the computations 
are appended below and to the right, and all the figures needed can 
be written into this one table (see Table 8). The method wiU 
converge to the least squares solution even when some cells are 
vacant or contain huge sampling errors. This is possible because 
the method may be used under any desired system of weighting. 

Directions follow for canying out the computations of the 
Stephan method in two dimensions when the weight of any ad- 
justed frequency is assumed to be inversely proportional to the 
corresponding sample frequency, as in the development of the 
normal equations (cf. Sec. 41). The numerical illustrations refer 

* Frederick E. Stephan, " An iterative method of adjusting sample frequency 
tables when expected marginal totals are known," Annals of Maihemaiicdl 
StaHstics, vol. XIII, No. 2, June 1942! pp. 166-178. 
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to Table 8 of this chapter, which is derived from the same samnlp ' 
frequencies as those in Table 1. ^ 

CICLB 1 

P‘(l) equal to m(,/2ra(.. Enter 
each factor m the proper row, one below the other, 1™ 

“ n “Stance, pji(l) = 2305/2 X 2^1 

= IS entered opposite the second row.) 

. Mulhply each sample frequency ntj in column j by the 
comspondmg factor pdl). These products are not needed 
mdiwdually they are to be accumulated in the product register 
of the machine until the vertical total for the column if ob- 

kI’ vertical total is 313 X 0.49791 

■f 156 X 0.60506 + 116 X 0.61701 + 1160 X 0 49707 + 154. 

X 0.49386 + 330 X 0.50366 = 1117.46423.) ThisSlJiot 
th?f transferred to the keyboard for 

the subtraction called for in the next step 

this accumulated total from the corresponding 
X cTum. ^ 2 and 3 fo? 


CYCLE 2 

ing\Sdir°ThZ^^® frequency in row i by its correspond- 
ing motor g^fi;, These products are not needed individuallv 
they axe to be accumulated in the product iSfS the 
machine unhl the horizontal total for the row is oSamed (In 

+ 26i X O?9O09 + iK n + 395 X 0.504K 

k not to he f ® ^ = 1186.53979.) This total 

tor too transferred to the key- 

f step. ^ 

?;SSiE5S«52 
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7. Repeat step 3 to get the factors g'j(2). (For instance, 
lm.4 - X P4(2 ))/ji.4 = {2213 - 1116.448701/2237 
= 0.49019 = 34(2).) 


CTCU! 3 

The process can be continued, that is, steps 6 to 7 can be 
repeated agoin and again. In practice, cycle 3 is often the 
last one. 


THE FINAL STEP; THE ADJUSTED TABLE 

The process will be stopped when another cycle would merely 
result in a repetition of the same factors. When this stage is 
reached, the factor pi + qj is formed and multiplied by the 
corresponding sample frequency inj, and the product is written 
in cell ij beneath the corresponding sample frequency mj. 
(For instance, for the ceU i = 3, j = 2, the adjusted sample 
frequency is 419(0.53386 + 0.60431) = 436, and this is written 
beneath the sample frequency 419.) 

In the illustration, there was no need of going beyond the 
second cycle, since, as will be observed, the p and q factors 
obtained in the third cycle are practically identical with those 
obtained in the second. But of course, one could not perceive 
this without going through the third cycle. 

It was mentioned earlier that the process is self-correcting. 
If a mistake is mode somewhere in the computations, the 
process will converge faster or slower, depending on the magni- 
tude and direction of the mistake. In consequence, fewer or 
more cycles will be required before the factors repeat themselves. 
The end result will nevertheless be the same as if no mistake 
had been made. The computer may therefore assume that 
when the factors repeat, bis work is correct, and he is ready 
for the final step. 

Since the Stephan method gives the least squares solution, the 
italicized figures in Table 8 (p. 124) are identical, except for round- 
ing errors, with the results in Table 1 (p. 107), which were obtained 
by the use of normal equations. The least squares results in both 
Tables 1 and 8 are in close agreement with those yielded by the 
method of iterative proportions in Table 6 (p. 118), and with the 
results to be obtained by the Bruy&e method (next aeotion) in 
Table 10 (p. 126). 

The choice between the different short-cuts (iterative proportions, 
Stephan, and BruySre) may reasonably lie in personal preference, 
though the Stephan method has certain theoretioal and practical 
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advantages, as mentioned above, and in u 

weigh heavily in its fayoj. situations these 

The Stephan method has been extended in n 
imenaiona; for general inatruotions see the 
m footnote 2 on page 121. ' inference to Stephan 


The Stephan adjustment 



7 = 1 

2 

3 

4 


362. 

781 

667 

313 


sen 

781 

660 

309 


mm 

395 

261^ 

156 


lB8t 

401 

861 

166 


1653 

419 

264 



1608 

4SB 

871 



10538 

2455 


1160 


10438 

8461 

1681 

1143 


1681 

353 

171 

164 


leee 

360 

167 

161 


3882 

867 

644 

339 


3914 

887 

64S 

338 

n., 

22847 


3493 

2237 

m,j 

88877 

6885 

3468 

8813 

Sf(l) 

).60134( 

O 

CO 

). 49099 ( 

),48973 

?/(2) 

a 

B 

M 


ff/(3) 

■ 

.50431 

.49084 

.49019 


1 





table 8 

applied to the example shown 


62740.49791 
BBsi 


2371 

ms 


23£2 


l£a69 

\um 


IN TABLE 1 


Pid) 


.505061 


.51701 


.49707 


2359| 

mo\ 


.49385 


5622 ,60366 


P<(2) 


Pi-(3) 


0.49680|0.49680| 
51011 


.53384 


.49420 


.48740 


.61011 


.63385 


.40426 


.48739 


.506991 


133837 

jS38S7 


.60699 


summed when 

totals (also italioized). The latter er' k* expected marginal 

than by direct addition rf 4^2," universe rler 

other two mav 

’ desci-ibed aa a precipitous 















[Ch. VII] 


ADJUSTING TO MARGINAL TOTALS 


125 


forcing at the end of the first half-cycle. It was shown to me by 
Dr. Paul T. Bruyfere, who had devised it some years earlier when 
he encountered the problem of adjusting sample frequencies in 
connejsion with some surveys in medical research. It does not 
give a least squares solution, but it is good enough, and has the 
advantage of being the most rapid of all methods here explained. 

1. Same as the first step in the method of iterative propor- 
tions (Sec. 46) : multiply the sample frequencies n,-,- in Bow i 
by the ratio mi./ni. as in Eq. 61. (This ratio will vary from 
tow to row.) Do this for every row. 

For illustration, this proportionate adjustment will be oairied 
out on Table 1 (p. 107), the result being Table 3 (p. 117). 

2. Form the column total m./. Subtract it (usually men- 
tally) from the known total m.#, and enter it as a “ vertical 
discrepancy ” along the top of a new table (Table 9) . Do this 
for every column. In the same way, form also the resulting 
horizontal discrepancies, rm. — mt/. (These would be zero 
except for errors in rounding off to integers.) 

3. Make up a table of corrections (Table 9), based on the 
vertical discrepancies found in step 2. Distribute any one of 
these discrepancies amongst the cells in that column, in propor- 
tion to the row totals, rm.. To do this, first calculate the ratios 
mi./n, where n is the total sample. Enter these ratios along the 
left of Table 9: they constitute the multipliers for fonning the 
final corrections, which are entered in the body of the table. 

The correction to be entered in row i and columny is the product 

I oi rm./n hy the discrepancy in column j. 

4. These corrections must now be forced, to equal the col- 
umnar discrepancies, exactly. This forcing is to be carried out 
BO that (o) the sum of the oorreotions in any row equals the 
corresponding “horizontal discrepancy,’’ written in step 2 
and entered in the right-hand column of Table 9, and so that 
(6) the sum of the corrections in any column equals the corre- 
sponding “ vertical discrepancy,” also written in step 2, and 
entered near the top of Table 9. Parts (a) and (b) are entirely 
independent. In Table 9 the forcing is indicated by putting 
parentheses around a figure obtained in step 3, and writing a 
new figure just to the left. Usually the forcing is small (a unit 
or so in any cell), and needs to be done in only a few cells, 
Large ceUs should be altered in the forcing, rather than small 
ones. (See page 84.) 
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TABLE 9 

CoBBBOnOMS FOB POnCtNG THE MAUQINAL TOTALS 

(BRiryiittE method) 

{Steps S, 3, and 4) 



TJli, 

n 

y = i 1 

31 

2 

Vertio 

(writ 

22 

3 

si discrepnnc 
ten in stop 2 
-33 

4 

es 

-22 

Bow 

Bums 

lift 

M 

1 = 1 

0.16S21 

5 

3 

-6 (-5) 

-3 

-1 (0) 

-1 

2 

.07078 

2 

BO 

-3 (-2) 

-1 (-2) 

-1 (0) 

-1 

3 

.07187 

2 

2 

-2 

-2 

0 

0 

4 

.46594 

16 (14) 

10 

-16 

-10 

0 (-1) 

0 

6 

.06886 

2 

2 

-2 

-2 

0 

B 


IB 

6 

4 

-5 (-6) 

-4 

0 (-1) 

n 

Column Bums 

BB 

22 (23) 

-33 (-32) 

-22 (-23) 




TABLE 10 

The PINAL HBSOTES OBTAINED DT THE BbOT&RE METHOD 


■■ 

n 

2 

3 

4 

ff 

mi. 

i 1 

3613 

781 

649 

300 

6262 

6252 

2 

1588 

400 

251 

166 

2395 

2396 

3 

1608 

436 

271 

118 

2432 

2482 

4 

10491 

2461 

1681 

1143 

16706 

16766 

6 

1662 

351 

167 

160 

2330 

2330 

6 

3915 

867 

643 

337 

6662 

5662 

W./ 

22877 

6286 

3462 

2213 

33837 

m.j 

22877 

6286 

3462 

2213 
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6. Add each forced correction to the corresponding frequency 
in Table 3. The result ia Table 10, which is the end product. 

Both row and column totals will agree with the controls, and 
the work is finished, except for multiplying all the frequencies m 
Table 10 by JV/n, to make them correspond with the popu- 
lation values (not shown here). 

60. Some remarks on the accuracy of an adjustment. A least 
squares adjustment of sampling results must be regarded as a 
systematic procedure for obtaining satisfaction of the conditions 
imposed, and at the same time effecting an improvement of the 
data in the sense of obtaining results of smaller variance tban the 
sample itself, under ideal conditions of sampling from a stable 
universe. As a matter of fact, the variance of the residuals arising 
in the adjusted cells will decrease with the difference between the 
total number of cells and the number of control totals, according 
to the results of Exercise 3 at the end of Chapter V 68). It 
must not be supposed that any particular adjusted cell frequency 
is necessarily better than the original sample frequency in the 
sense of being closer to the complete count. It may be, but also 
it may not be, and there is no statistical way of discovering which. 
All we know is that on ihe average the adjusted cell frequencies 
will be better. 

But the decrease in variance is not all; adjustment to known 
control totals has at the same time the effect of ehminating biases 
m the nature of inherent differences between the sample and com- 
plete count. This effect is often more important than the decrease 
m the variance of the sample frequencies. 

It is desirable to get some idea of the errors of sampling by actual 
trial, such as by a comparison of certain sampling marginal totals 
with the corresponding universe totals, as can often be arranged by 
means of controls. Also, the sample can be tested for regularity 
of patterns. There is another aspect to the problem of error — 
even a 100 percent count is not by itself useful for formulating 
social and economic plans, except so far as we can assert on other 
grounds what secular changes are taking place. 
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CHAPTER VIII 

CURVE BITTING IN MORE COMPLICATED CIRCDM 

STANCES 

61. Some general remarks on the purpose of curve fitting. To 
^end the theme of Chapter I, we may say that the reason for 
fitting a cum to a set of data is to summarize the evidence pro- 
vided by that experiment for maJcing predictions with regard to 

It IS ihi data of the next experiment that one holds in awe. Will the 
curve fit, or will it not? And when we decide whether the cum 
fits, we do so on the basis of whether it fits well enough to give 

Axa deduction. 

^ out m praotioe dosdy enough w that it tan be uood o» a 

"ado from a 

i"™" “f have made it unneoosmy 
^(moertamoteenperimente. As an motaneo, wo nuy 
E^ple 1 of Chapter XI, where a quartic is fitted to some 
^preosiMty data publishtd by the Micbels in Amsterdam 
^ Warbj when fitted to their oomp«jbility dawS^ 
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poses, wedidim is satisfactory in place of the Joule-Thomson 
coefadent that would be observed directly. Certain checks, in 
terms of other experience and other deductions, which may be 
available in isolated portions of the ranges of pressure, volume, and 
temperature that are covered by the compressibility data, ’lend 
confidence to the results — confidence of the land that can be 
translated into action, such as the design of compressors for refrig- 
erators and other machinery. 

In order to extend the region of prediction into other areas and 
other ranges (e.g., other cities, other economic levels, higher pres- 
sures, higher temperatures), not yet included in the experiments, 
it is necessary to have related experience, or meanwhile to regard 
extrapolations as pure conjectures. Such conjectures may be 
regarded as predictions, but without a high degree of belief, and 
not as a scientific basis for action. 

It is important to keep in mind the ultimate purpose of curve 
fitting, particularly when one is actually fitting curves for purposes 
of action. Meanwhile, it is necessary that one learn to perform or 
understand some of the procedures by which curves can be fitted. 
To this end, we resume our study of the adjustment of observations, 
returning to the general solution worked out in Chapter IV. ’ 

It will be recalled that earlier in the book some simple problems 
in curve fitting were treated (Secs. 9 and 10, the single sample; 
Sec. 12, several samples ; Sec. 15, a line through the origin). These 
problems were simple, not just because the functions were simple 
ones, but also because the errors in the variables entered in such 
manner that the parameters (adjustable constants) could be found 
directly by differentiating /S. The solutions obtained for those 
circumstances were and still are satisfactory, but the research 
worker must be prepared for more complicated situations, such as 
both variables subject to error, or functions in which the param- 
eters and the errors do not enter in so simple a manner. 

A framework will be developed in this chapter for more com- 
plicated circumstances. The simple problems just mentioned will 
of course fit into this framework, as will occasionally be pointed out. 
Fortunately, the solution is already worked out in general terms; 
it is contained in the general normal equations on page 56. All 
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we need to do now is to apply this solution to curve fitting and son 
how It con be adapted to routine computational procedure 
There will be a function to be fitted. We might write it 


y‘, tt, b,c)=0 


( 1 ) 


to indicate that there is an equation involving x and y, and the 
(Klji^e) a, b, apd c. In the exempli aiJS 

seen in Selena 9, 12, and 16, the Imclionn were simple, namely 
* = a, and j/ = 6a;. ^ 

(constituting 

Part C) did not contain parameters; the conditions were rigorous 
bemg geometac, or forced by complete counts. No question arose 
Mncei^g the adjustment of parameters, because there was none 
We threw away with abandon certain rows and columns of the 
^neral solution that owed their origin to the parameters (p 69) 
Now, however, we can not do this; the condition equations contain 

that there will be simplifications of other kinds, and we ahlu 
a procedure not unhke that contained in ChapterT V 

62. Graphical considerations. It is desirable to have in mind 

• points, calculated points^and true 

c^fand^he“to^^*"'^f7*'’ “o on the calculated 

c^e and the true pomto he on the true curve. The equation of 

the i^lculated curve may be written in the form of Eq 1 wherem 

® and y are the coordinates of any point on the curve, and a 6 c 

m fobetS1?ii" ““ P««nietem. wuil, 

were oeeliiibfe ’the “ “^ed^t if the errore of obsaivotioL 
e negligible, the observed coordmates X and Y would satisfy 
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As in Chapter IV, we shall need some approximate parameters 
do, bo, Co to start off with. If they were used for drawing the curve 
in place of a, b, c, there would be still another curve (not shown) 
in Figs. 16 and 17, which might be called the approximuts cutvb 
if it needed a name. 

The calculated coordinates, and the calculated parameters, 
satisfy Eq. 1 exactly. The observed coordinates, however, and 
the approximate parameters Oq, bo, co, do not. When, for any 
pomt, the observed coordinates X and Y, and the approximate 
parameters, are substituted into the left-hand side of Eq. 1 the 
equation is usually not satisfied, which is to say that the left-hand 
side is usually not zero, but is instead some sTnall quantity Fq, 
defined as 

Fq = F(X, Y] oo, bo, Co) (Cf. Eq. 5, p. 62.) (2) 

Of course, Fq may by accident be zero at some point, and will be 
, zero by design at any point through which the approximate curve 
is forced to pass, as when the method of selected points is used to 
determine satisfactory approximate values Oo, bo, Co (cf . the reduced 
type at the end of Sec. 65). The quantities Fq at point No. 1, 
No. 2, etc., will appear in the normal equations of Section 56. 

For Bimplioity and definiteness, the development will be written 
out for only two coordinates, x and y, at each point. The extension 
to three coordinates is obvious, in which event Eq. 1, instead of 
being the equation of a curve in the x, y plane, is written as the 
aur}wx F {x, y,z\ a,b,c) = 0 in the a, y, z space. See Examples on 
pp. 231 ff. for an illustration in three dimensions, and Exercise 26 of 
Section 71 for one in four dimensions. An increase in the number 
of dimensions does not necessarily increase the complexity of a 
problem. 

A point is observed to be X, F; that is, the x coordinate of some 
tme point {, is measured, perhaps several times, and the mean of 
these measurements is X with weight Wx. Likewise, the y coordi- 
nate of the same (true) point is measured, perhaps several times, 
and the mean of them is F with weight Wy. By Eq. 16 on page 22, 
the weights of the observed coordinates X and Y at this point will 
be in the inverse ratio of the variances of protracted random series 
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of meafluremeats on the true coordinates f, ij (Gh. I), and directly 
in the ratio of the numbers of observations taken. Otherwise 
expressed, if these variances are denoted by Var x and Var y, and 
the numbers of observations by Nx and Ny, then the ratio of the 
weights will be 




Nx Vary 
Nj, Varx 


( 3 ) 



Fiq. Id. A typical aituatioa in curve fitting. It is assumed that the " true 
points," wherever they are, lie on the “true ciuve ” P(x,y; a, jS, y) - 
0, a, 0 , 7 being the true and unknown values of tho parameters. The *' cal- 
culated points" all lie on the "caloulnted curve" F{x, y, a, h, c) = 0, o, b, c 
being the calculated values of the parameters. This figure and the next one 
first appeared in an article entitled "On tho chi-test and curve fitting," 
J. Amer, Slat. Amo,, vol. 29, 1934: pp. 372-382. 

Of course, this ratio may vary from point to point, depending on 
the variation of the factors on the right (cf. Example 2, pp. 218- 
230). 

63. The conditions. For each point observed, there is a cal- 
culated point, and this calculated point is forced to lie on the 
calculated curve (preceding section). The residuals must be just 
the distances required to put the calculated point on the calculated 
curve; see Fip. 16 and 17. Now since the calculated point must 
lie on the calculated curve, its coordinates must satisfy Eq. 1 
(p. 130), which is to say that 

F(x, y, a, h, c) 

must vanish at every one of the calculated points, as indeed it must 
at all points along the calculated curve. Thus, for every point 
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there is one condition imposed on the adjusted coordinates. 
Altogether, there are as many conditions as there are points. Por 
n points there are n conditions. 

Next, we look at the general normal equations {p. 56), also at 
the conditions that were imposed (Eqs. 3, p. 50), and we seek a 

CALC'D CURVE 



Fifl. 17. Relations between the "true,” " observed,’' and " coloiilated " 
points. The x and y coordinates of a point are observedj these observations 
when plotted give the ** observed points.” The point that was measured 
is the true point, * which is unknown and lies on the unknown " true curve ” 
F(x, y; a, p, y) = 0, a, p, y being the true but unknown values of the param- 
eters. The calculated curve ” is found by adjusting a series of observed 
pointsj its equation will have the same form as the true curve, but the param- 
eters therein will be the calculated parameters” c, &, c. Oorrcsponding to 
each observed point there will be a " calculated point,” whose coordinates 
are found by subtracting the “ residuals " 7» and 7„ from the observed co- 
ordinates X and Y, Ex and Ey denote the " errors in the observed points 
Fy, and 17® and Uy are unknown, but and Vy are calculated along with 
the parameters a, 6, c by the method of least squares. As the figure happens to 
be drawn, each of the six quantities B^, Ey, U®, Vy, 7®, 7„ is positive. Their 
signs are indicated by the directions of the arrows. 

way of writing these conditions so as to force the calculated points 
to lie on the calculated curve. This can be done by writing the 
condition functions in the form 

P* = F(xh, Vh] a, b,c) h = 1,2, ■■ -,71 (4) 

wherein the function F on the right is the function found in Eq. 1, 
which is to be fitted, and Xh, Vh are the final calculated or adjusted 
coordinates at point h. These coordinates, along with the cal- 
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culated parameters a, B, and c, are to have such values that the 
function F on the right vaniehea at every calculated point. 

64. The L coefficients. Immediately upon writing the condi- 
tion functions in this way, we perceive that the coordinates of 
point h will enter the condition function F{xk, y*; a, b, c) for that 
point, but not the condition function F(xg, yg] a, b, c) for some 
other point g. As a consequence/ 


d 

T-F(xh, yh] a, b, c) = 0 

aXff 

a 

yh\ a, B, c) = 0 

ayg 


if g 9^ h 


( 5 ) 


It then follows from Eq. 14, page 55 (wherein L was defined for 
the general solution), that 


Lgh = 0 when g h 


( 6 ) 


This means that the L coefficients in the general normal equations 
(p. 55) standing off the diagonal are zero.^ Moreover, the L 
coefficients standing on the diagonal contain only two terms each 

VkAAA*..... ..11 J.1.^ _1 . ■ 1 « 


write 

Po = ■^^P(.x^,yh; a,b,c) 

and 

= J^JP{y>h,yh-, a,b,c) 

whereupon 


n 

«S 


Wx Wy 


( 7 ) 


( 8 ) 

■^e suffix h has been omitted, it being simpler to leave it under- 
stood that the derivatives and the weights, and hence the L coeffi- 
cients, may vary from point to point. 

^ ooordinatea o! one point do enter the condition 
function for an adjacent point waa published by the author in the Phil. Mag., 

iir oscillations of ^e 

pointer on a chemical balance. 
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Remark 1. The first term on the right of Eq. 8 drops out at 
any point where X is free of error, for then the denommator 
is « ; and similarly the second term drops out if y is free of 
error. If both X and Y are subject to error, the two terms may 
be of comparable magnitude, and both accordingly retained. 

(Cf. Remark 3 in Exercise 4 of Sec. 66, p. 181.) 

Remark 2. From the way in which the L coefficients enter 
the normal equations, and affect the standard errors of the 
parameters (Sec. 62), it will be seen that one object in the dwign 
of an experiment should be to produce small values of L. The 
two terms on the right of Eq. 8 give an indication of where timn 
and funds may wisely be apportioned. If one is already 
small (possibly a quarter) compared with the other, then it 
might not be worth the necessary expenditure to reduce that 
term, already small, to half its value. Better it would be to 
spend even more time and funds to halve the larger term. Sim- 
ilarly, if the L coefficient at one point is already small compared 
with the L coefficients at some of the other points, then iTiatAgd 
of using time and funds to reduce the small one further, it 
might be wiser first to reduce the L coefficients at those points 
where they are largest. 

Remark 3. By Eq. 9, page 40, we see that the L coefficient 
at a particular point is none other than the reciprocal of the 
weight of F evaluated with the corresponding observed coordi- 
nates X, Y. This value of F would be written F(Z, 7; a,b,c). 

It is the quantity designated as Fq' in Exercises 3, 4, and 5 of 
Section 58, pages 145-146. Since L is the reciprocal of a weight, 

1/W will frequently be written in place of L as we go along. 

(Cf. Remark 3, p. 181.) 

56. The normal equations for curve fitting. The general normal 
equations (p. 55) now take the form shown below. These can 
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quicWy be reduced to a smaller set, in number equal to the number 
of adjustable parameters. First, eliminate Ai, ■ ■ . X„bvX? 

for them m the upper n equations, getting ’ ^ 

= Fa^A - Fi^B - F^^C) 

^3 = ;^ iF<? - F^^A - Pi^B - F„^C) 

(A X for 

each point) (lo) 

I 

“ 2^ (^o“ - Fa'*A - Pi^'B - F,^C) 

J 

Then substitute these values of Xi X» • • • ^ I'nf.x+'u i 

r»™ of Bqo. 9. The «sult t 'll. ' t ta 

p. 69), the coefficients below tliA ^ i * ^ .. . . 


A B 

c 


1 

[¥] [^?] 
m 

m 

[ t ] 

m 


m 

["?] 

[^] 


(The normal 
equations for (11) 
curve fitting) 


positive definite, like the general norm'ni ^ 

came (Sec 28) Onpo ti, equations whence they 

valu« o, S, nomd equation, the adjueW 
tion. More explicitly, ^ ™°iediately by subtrac- 


tion. More explicitly, 

0 = ao - 4 

1 = h~B 
c= Co - C 


(Eqs. 6, p. 52) 


a. 6, and c have^ilLe^ted^ adjusted values of 



[Ch. VIII] 


OimVE FITTING 


137 


Several details remain: to adjust the observations (Secs. 66 
and 68 ); to work out a systematic procedure for forming the 
normal equations and solving them (Secs. 60 and 61); to discover 
in this systematic procedure a quick way of calculating the mini- 
mized sum of squares, 8 (p. 57 and the exercise following); to 
calculate the variance and product variance coefficients of a, h, c 
(found in the reciprocal matrix, Secs. 61 and 62). 

It is important to observe that the final values of a, 6 , and 
c will be independent of the approximations od, 60 , co- That is 
to say, two computers, starting off with slightly different ap- 
proximations (but with the same observations), will find theii 
parameter-residuals A, B, and C to be just enough different so 
that their final calculated values of a, b, and c, and hence their 
final calculated curves, are practically identical. 

Under some circumstances, and for some purposes, however, 
the approximations ao, ho, co must not be too rough. In other 
problems it makes no difference what these approximations 
are, except that alwa 3 rs the rougher they are, the more figures 
are required in the normal equations, hence the greater the 
computational effort required. These remarks are repeated 
more specifically in Exercises 4, 6 , and 10 in Chapter X (pp. 179, 

183, and 187). 

In regard to the matter of arriving at the approximations 
ag, hg, cg, it should be made clear at the outset that in practice 
this is usually not difficult. Often one will have good enough 
approximations simply from previous experience. There are 
graphical methods, by which one draws in a curve free-hand, 
after malting a judicious choice of scales, such as changing 
y = ae*® into the logarithmic form In j/ = In o -f 6 *, to make 
it straight. There is the “method of averages,” called by 
Norman Campbell* the “ method of zero sum," by which one 
finds what values of a, b, and c will force the calculated curve to 

* Norman CampboU, PhU. Mag., voL 30, 1020: pp. 177-104. See also 
Whittaker and Bobinson’s Calcuius of ObservoMme, Art. 131, p. 268. Ac- 
cording to them, the method of averages (i) was much used in the latter 
half of the 18th century; (ii) was first published by Tobias Mayer in 1748 
and 1760. A recent paper by Wald contains some interesting and valuable 
theoretical work on the method of averages. It turns out that when the x 
and V observations have weights in constant ratio, the method of averages is 
unbiased, and in statistical efficiency compares well with the method of least 
squams, at a considerable saving in labor (in agreement with Campbell). 
The reference to Wald’s work is the AnmU of Math. Statistics, vol. 11, 1940: 
pp. 284-300. 
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av^e out correctly over groups of points (three Erouna if 
there are three parameters). Then there is Cauchy’s 
which has much to recommend it. ^ method, 

Lastly, there is the so-called method of selected nninfc 
cernu|which brief mention was inade at the end' of SeSn S' 
page 52. By this method one simply selects three n/i; + ' 

usually two end points and a middle point if the™ ~ 

through these pomts. The vidues so found serve as L 6 T 
H there are two parameters, two points are selected ° 

This calculation is often fairly simple to oarrv out nrid ■+° 

fipJnn points B much used and easily iusti 

fled on grounds of simplicity, yet it is ahnnt +i,» ™ ^ ^ . 

able method of curve fittiu/ if conceiv- 

CM be said, if there wore no errors in anv nfltl • 1 
yidd the correct results for a, b, and c Pomts, it would 

ii^ residual at point h) (12®) 

F - ^ 

(j/ residual at point A) (122,) 

Once the residuals and F hn™ u 
®fc - Zft — Fa, 

s, . n _ («• 6, P- 52.) 

•C«u*y, (!«,toro*s,rt.2t, 1847;p. Ml 
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The numerical value of the Lagrange multiplier X* can be 
found from Eq. 10 (p. 136). E* and are numerical values of 
the derivatives of F at point h. After differentiation, F* and Fy 
are functions of x, y, and a, b, c. We can not evaluate t.liq se 
derivatives numerically at the calculated pomt h until we find 
the residuals and the calculated coordinates, but that is just 
what we are trying to do now. In practice, fortunately, for use 
in Eqs. 10 and 12 it is sufiioient to evaluate the derivatives at 
the observed point, using the approximate parameters, though 
the final calculated parameters can be used if desired. 

Eqs. 13 give the coordinates of the calculated point correspond- 
ing to the observed point X^, Yh. Eitiding the calculated points 
is the process of adjusting the observations; when Xh and yh have 
been calculated, the observations Z;„ 7* are said to be adjusted. 
The calculated point Xh, yh is the least squares estimate of the posi- 
tion of the unknovm true point fj, rj;,. Obviously, it will depend 
not only upon Xh, Yh, and their weights, but also more or less upon 
all the other points and their weights. In randomness, the 
variances of the calculated coordinates are less than the variances 
of the observed coordinates. 

Just how is this dependence tied up with the other points? 
Through the normal equations (Eqs. 9, p. 136), or their equivalent, 
Eqs. 10 and 11. Eqs. 11 supply the parameter-residuals A, B, 
and C, which are to be used in Eqs. 10 to find Xi, X 2 , ■ ■ •, X„. 
These in turn are used in Eqs. 12 to compute the x and y residuals 
at each point, by which the observations are adjusted, as indicated 
in Eqs. 13. 

We now have a method of adjusting the observations and of 
estimating the parameters a, b, c, when both the x and y coordinates 
are subject to error, but it must be remembered that the solution 
depends on certain simplifying assumptions; namely, that the 
squEires and higher powers of the residuals can be neglected in the 
Taylor series of Chapter IV. 

B&rnrk 1 . A familiar method of adjusting the observations, 
valid when all the x coordinates are free of error, is to substitute 
the coordinate free of error into the formula F{x,y; a, b, c) = 0 
(Eq. 1 ), and solve for the other coordinate. Thus in the parab- 
ola. 


y = a + bx + cx'^ 


(14) 
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if * is free of error, it is easy to oaloulate y for a given a:, once 
a, b, and o are determined. But if a: is subject to error, one 
must either solve a quadratio for s in terms of a known y, or — 
what is usually easier — adjust the x coordinate by using 
Bqs. 12 and 13, after evaluating the Lagrange multipliers in 
Eqs. 10. 

When the function F of Eq. 1 is not solved for y explicitly, 
it may be easier to adjust the y coordinates by means of Eqs. 10, 

12, and 13, rather than to substitute directly into Eq. 1 and solve 
for y in terms of x. Similar remarks apply for adjusting the 
coordinates when x alone is subject to error. 

When both coordinates are subject to error, one must apply 
Eqs. 10, 12, and 13, if he would adjust the observations. (For 
a numerioal illustration, see the mmmple treated in Section 78, 
pages 227 £f. As an exercise, the student could at this time 
work out the numerioal values of the remaining nine calculated 
points in that example.) 

Remark S. Gauss and others gave methods for adjusting, 
the observations in problems of geodesy and astronomy 
(Chs. V, VI, and VII, constituting Part C). Unfortunately, 
they did not give much attention to problems in which the 
conditions contain parameters (curve fitting), especially when 
more than one coordinate is subject to error. 

It has BometimeB been said that least squares is reasonable 
enough in surveying and astronomy, but that it is illogical and 
equivocal in curve fitting. Actually, the principle of least 
squares is always the same (p.l4). The distinction between the 
problems lies in the conditions that the adjusted quantities are 
subjected to.' A neglected but worthy paper by KummelP in 
1879, had it not been overlooked, could have set matters 
straight. Later papers by Stewart® (1020) and Uhler“ (1923) 
also emphasised the unity of the different Icinds of problems. 

Bemwrk 3. The term “adjustment of observations,” as 
it has often been applied heretofore to curve fitting, has meant 
a oaloulation of the parameters o, b, c from a set of data. Now 
we see that the parameters enter only as unknowns in the 
conditions that are forced upon the adjusted quantities. Least 
squares is primarily a method of adjusting the observations, 
and the parameters enter only incidentally. As a matter of 
fact, least squares is the only method of curve fitting by which 

* Charles H. Eummell, The Analyst (Des Moines), vol. 6, 1879: pp. 97-106. 

Meldrum Stewart, PhU, Mag., vol. 40, 1920: pp. 217-227. 

'HoittM S. Uhler, J. Optical Sac., voL 7, 1923: pp. 1043-66. 
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one can profess to odjuBt his observations. But, of course, the 
determination of useful values of the parameters and their 
standard errors is often the prime purpose of an investigation. 

Remark 4- The method of least squares is the only analytio 
device for curve fitting that takes account of the weights of 
the observations. If both x and y are subject to error, it is 
necessary that both coordinate be given their proper weight- 
ing at every point. In graphical methods of curve fitting, the 
eye can be trained, to some extent, to take account of weights. 

Remark 5. In the next chapter we shall see that it is possible 
to compute 

<S = 2 (16) 

without calculating the individual x and y residuals, nor squar- 
ing, weighting, and adding them. However, it is often found 
worth while to draw the calculated curve, and to lay off the x 
and y residuals, to bo able to note whether pny of them is espe- 
cially large. In this manner, it is sometimes possible to discover 
sources of spurious observations that would be hidden in a com- 
prehensive test like the chi-test. 

67. The distribution of The least squares value of 

= + (16) 

for a fitted curve (provided it is the right curve, and the observa- 
tions are random) has the probability distribution^ 

(17) 

wherein 

k = number of points — number of adjustable parameters (18) 

k is commonly called the “ degrees of freedom.” It was recognized 
by Gauss, though he gave it no particular name (cf . footnote 6 in 
Ch. II). The effect of including both x and y residuals in x* is 
merely to add the second term in the summation in Eq. 16. The 
form of the distribution is unaffected. 

^ See an article by the author in the /. Amer. Stal, Assoc,, vol. 29, 1034: 
pp. 372-382. A necessary lemma thereto is given in the PhU. Mag., vol. 19, 
1936: pp. 389-402. 
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Historical mk. The distribution of x* for problems in curve 
fitting where y alone is subject to error was first published by 
P. Pizzetti in an article entitled “ I fondamenti matematici pm 
la critica dei risultati sperimentali,” Atti Mia Regia VniverrilA di 
Genova, vol. xi, 1892: pp. 113-333. Plelmert’s distribution of 
s for the simplest problem in cuiwe fitting (Sec. 9), arrived at 
also many years later by Student in 1908, can easily be converted 
into the distribution of Helmert gave this disti-ibution in 
Sohiemilch's Zeilschrtft filr MaOi. und Physik, vol. 21, 1876: 
pp. 300-3, and it is interesting to note that Pizzetti referred to 
Helmert’s work. Helmert’a derivation is reproduced in 
Emanuel Czuber’s Bedbachlwngsfehler (Teubner, 1891), pp. 147 - 
150. Pizzetti’s result was generalized by the author for eiTors 
in both the x and y coordinates, in the paper referred to in 
footnote 7. The assumption of normally distributed observar 
tions was presumed. 

There is no such thing as a distribution of unloag the 
fitting is done by least squares; in other words, only the mini- 
mized x“ has a distribution. 

68. Some geometry concerning the adjustment of observations. 
Now let us consider some of the details connected with the calcu- 
lated points, or the adjusted observations. Let Q be the line 
accent joining the observed and calculated points. By Eqs. 12 
'wejsm. find the slope of this line segment; it is 


dF_ 

Slopeof Q = ^ residual 

the a: residual v}yF^~WydF~ Wy^ 

dx 


( 19 ) 

The last step here involves the very important relation learned 

in elementary caloulus, that if P(z, y) = o, then— = _ ~ 

dx dP F,i‘ 


Eq, 19 says that 


dy 


The slope of Q = — — ^ 

Wy the slope of the curve 


(20) 
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Hence if w® = % at pomt h, the two slopes are negative reciprocals 
of one another, ^d, if the a: and y scales are equal, the line segment Q 
mil be perperuiicular to Ike curve (but see Exercise 1 at the end of 
this section). 

If at any point, = « or is very large, which is to say 

that X is relatively infallible, then the line segment Q is vertical 
and the adjustment is all in the y coordinate. An exception may 
occur in the neighborhood of any portion of the curve that is 
vertical or nearly so; there the derivatives f * and Fy usually affect 
the normal equations and Eq. 20 in such a way that the curve is 
brought close to the observed point, and the line segment Q is 
drawn away from the vertical. 



Pia. 18. A portion of Fig. 17 redrawn for further p.nT»iiHBr n. f,| on 

If at any point, % : w* = oo or is very large, which is to say that 
Y is relatively infallible, then the line segment Q is horizontal and 
the adjustment is all in the » coordinate. An exception may occur 
in the neighborhood of any portion of the curve that is horizontal 
or nearly so; there the derivatives Fx and Fy usually affect the 
normal equations and Eq. 20 in such a way that the curve is brought 
close to the observed point, and the line segment Q is drawn away 
from the horizontal. 

If at any point, Wx : iw,/ is finite, i.e., both X and Y are subject to 
error, the line segment Q will be neither horizontal nor vertical, 
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[Sac. fig] 


but mclmed, and there is adjustment in both the x and y ooordi 
nates. Exceptions occur. In the neighborhood of any verticd 
portion of the curve, the line segment Q may be pulled to a nearly 
horizontal position, and, in the neighborhood of any horizontal 

portion of the curve, the line segment Q may be pulled to a nearly 
vertical nosition v 


Exnitcisss 

teise 1. Suppose that the a: and y scales are equal on the 
graph of a certain curve, and that, at one of the points, the weights 
/ j!?- ^ ^ 6qual, wherefore the line segment Q in Eig. is 

“ perpendicular to the curve at that point. Then supnose 
that ae graph is redrawn, and that the units in which Y is measured 
are changed, as from feet to inches, while the units of X remain 
unchanged. 

If a'*''® *^''* *?^ ® perpendicular to 

the fitted curve. {Hmt: The ratio : w, was unity before the 

change m aede, but it is not so afterward. If all the y coordinates 
Me multiphed by C, because of the change in units, then the weights 
of all 1/ observations are decreased by the factor l/C®, and the mw 
vdue of % is C times the old one. Moreover, the new slope 
of the curve is C times as great os before. By Eq. 20, the slope of 

tnat y IS no longer perpendicular to the curve.) 

(J) Show that the change in the units of measuring Y affects 

£ ^ y coordinates of 

t Wculated curve and the adjusted points are aU multiplied by C 

“ automatically taken care of by the 

cu^1i+T ® arbitrariness of 

ZSw ® results may arise 

merely from a change m units.) ^ 

Exercise S. When there are three coordinates, the surface 

y, z\ a, &, c) = 0 

tems-S?? ^ I ^ three 

tW ^ ® Ww,. Show 

that If X, y, z are observed with equal weight at any point, the line 
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segment Q joining the observed and calculated points is normal to 
the fitted surface.® In such a problenij the calculated points lie on 
the calculated (fitted) surface. See Section 81 for an example in 
three dimensions, and Exercise 26 of Section 71 for one in four 
dimensions. 


Exercise S. Show that the minimized sum of squares, S, can be 
written as 2 WFq’^, wherein Fo' is the left-hand side of Eq. 1 
(p. 130) evaluated with the observed coordinates X, Y, and the 
calculated parameters a, i, c\ and W is the weight of Fq (of. 
Rem ark 3 at the end of Sec. 54), 


and 


From the way Fo' and W are defined, it turns out that 
Fq' = F(.X, Y; a, b, c) 

= Fo ~ FttA — FhB — FeC (Ne^ecting residuals 

of higher order) 

= F,7„ + FyVy (See Eq. 7, p. 53.) (21) 


W = j at point h 


( 8 ) 


The solution to the problem then Hes in writing, as usual, 

S = 2 (w*Fx* + %7,») 

and noting that from Eqs. 10 (p. 136) 

X = WFo' at point h (22) 


whence Eqs. 12 give 


7.= 
Vv = 


1 I 

— WFa'F, 

Wx 

— WFs% 

Wy "J 


(23) 


for the X and y residuals at point h. Substitution into WmVa^ 
+ WyVy’^ gives the required result in terms of Fo' (due to Kum- 
mell, 1879). This result is useful in Exercise 3 of Section 61, 
page 163. 


Exercise 4- Prove that W in the preceding exercise is actually 
the weight of Fq, i.e., of F{^X, Y] a,b, c). (Hint: Apply Eq. 9, 


° This result was proved by the author in the PhU. Mag., vol. 11, 1931: pp. 
146-166. 
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p. 40.) Hence the new expression L WFq'^ for S can be regarded 
as a sum of the weighted squares of residuals, Fq now being defined 
as a new kind of residual. 


Exercise B. If the x coordinate is free of error, WFq'^ is equal to 
and, if 7 is free of error, WFq'^ is equal to w^Vx^. 

Exercise 6. Prove that for any observed pomt in the neighbor- 
hood of which the slope of the fitted curve is positive, the residuals 
Vx and Vy will have opposite signs; but, if the slope is negative 
then 7* and Vy will have the same sign. In other words, when 
the fitted curve lies below the observed point, then the calculated 
point lies below and to the right of the observed point if the slope 
of the fitted curve is positive, but below and to the left if the slope 
is negative; and when the fitted curve lies above the observed 
point, then the calculated point lies above and to the left if the 
dope^ positive, above and to the right if the slope is negative 



point. 


Exercise 7. Show that in fitting 


y = a + hx + cx^ 

with y alone subject to error, Eqs. 10 reduce to 

X = «)„{7 - (oo + &oa: + c^x^) - {A + Bx + Gx^)] 

= Wv(7 - y) 


(24) 

(25) 

(26) 



[Ch. VIII] CURVE HTTING I 47 

whence Eqs. 12 and 13 on page 138 reduce to 



= 7 - {y - (fflo + M + CoS®) - iA+Bx + (7®2)j (27) 
or 

y = a + bx + cx^ 

In this circumstance, therefore, y may be calculated at any point 
merely by substituting the x coordinate into the equation with the 
adjusted values of a, b, c. 



CHAPTER IX 


SYSTEMATIC COMPUTATION FOR FITTING 
CURVES BY LEAST SQUARES 

69, Frelisiinaiy note OQ the tabular soluUoa. In the systematic 
solution of the normal equations for geometric conditions, given 
on pages 82 and 83, -we saw an easy way of conaputing the mini- 
mized sum of squares, S. In Section 61, we shall see that the same 
routine for solving the noimal equations in curve fitting will also 
yield S. We shall, moreover, see how our initial approximation 
to the sum of squares is diminished as one parameter after another 
is adjusted. 

In order to gain some preliminary familiarity with these charac- 
teristics of the routine, we shall return to the simple illustration 
considered in Section JO, where we had the n observations and 


ObbBBVATIONB and waiQHTB (COIDMNB 1 AND 2) 
CoMFVIAHONB FOH FINDING X AND g (cOIiUMNB 3 AND 4) 


a) 

Observation 

(2) 

Weight 

(8) 

Weighted deviation 
from ao 

(4) 

Weighted square 
of the deviation 
from Ofl 

ai 

lOl 

- Oo) 

MlfEl - do)® 

32 

Wi 

W2(S8 - Oq) 

Wi(jCi - ao)“ 

*8 

Ms 

Msfss - Oo) 

m{H - do)® 



Mn - do) 

lOnfrn - Oo)® 



rrr, , £ M(J! - Ofl)® 

Wtd. av. ■- 





= p 

e Q 
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weights listed in columns 1 and 2 of the table on page 148. The 
weights could be merely the relative frequencies of occurrence. 

The problem now is the same as it was in Section 10, namely, to 
find what value of the parameter a in the equation 

* = “ ( 1 ) 

renders the (weighted) sum of squares, S, a muiiTnuTYi Now, 
however, we shall start off differently, because we wish to pattern 
the solution after the framework to be explained in Section 61 for 
more complicated problems. We shall use an approximation oq 
for a, and shall correct it later by finding the residual A, which, 
when subtracted from oq, gives the final value of a (turn back to 
Eqs. 6 on p. 52). 

Corresponding to Eqs. 11 in Section 65 (the normal equations 
for curve fitting), there will here be one and only one normal equar 
tion, with one unknown. We proceed to calculate the one and 
only L coeflacient therein; also the right-hand member. 

For the present problem we set 

F==x -a (Eq. 1 of Ch. VIII) 

whereupon 

Fo = «7i — Oo (For observation No. h) 

The derivatives are 

F, = 1, Fa = -1 (Turn to Eq. 6 of Ch. VIII.) 

whence 

L = ^ = - (Eq. 8 of Ch. VIII) 

Wx W 

The right-hand member of the one and only normal equation will be 

^ ~ ~ «o) 

The sum of squares formed with oq will be 

i:>(a: - oo)2 
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which, be it noted, can be written as [FqFq/L\, a symbol that in 
more complicated problems denotes the sum of squares formed with 
the approximate values (oo, &o, cq) of the parameters. In geo. 
tion 61 and beyond, this symbol is abbreviated to [oo]. 

We now male substitutions into Eqs. 11 on page 136. Row I 
constitutes the one and only normal equation. For reasons that 
may become clear later, we introduce Row 2, containing in the 
“ 1 ” column the sum of squares formed with the approximation oq. 
Rows 3 and II are formed by the manipulations described in the 
column " How obtained.” 


Row 

A = I 


I 

Z “ — ' Z — Oo) 


2 

Z ~ oo)* 

How obtained 

3 

- IZwlai-oolp/Z"' 

Multiply I by 

11 

Z w(x - ao)* 

+ Z w(® ~ oo)/2) w 

Add 2 and 3 


-- {Z«»(®-oo))VZ 

w 


Solving Row I for A, we get 


^ _ Z - gp) 

Z w 

whereupon 

a = do - 4 (Gf. Eqs. 6, p. 52.) 

Z - Oq) 

Z W 


— Oo + 


1 

- Oo + Oo 

Zmi 


T,WX 

'Zw 


= 2 


(By definition of x) 


( 2 ) 


( 3 ) 


least squares value for o is thus x, the weighted mean of the 
ob^rvations, as was obtained by the direct solution in Section 10a. 
The extreme left entry in Row II is none other than S, the 
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minimized sum of squares. This is so because 


(The extreme left entry 


£ w 


= 2 


2 _ 


iZwx? 

2 w 


= 2 wx^ — 5® £ 2i) (4) 

which is the value of S shown by Eq. 11 in Section lOo, page 19. 


This tabular solution will be extended later in this chapter to the 
calculation of parameter residuals (called A, B, C) and the mini- 
mized sum of squares, S, in more complicated problems in curve 
fitting. 


At this point, it is interesting to perceive that the tabular solu- 
tion just described is equivalent to a certain rapid method, often 
used in statistics, for computmg the mean x and the standard devi- 
ation 5 of a set of n observations such as those shown in column 1 
above. The method will be described in steps. 


i. Select an arbitrary datum, perhaps a rounded-off guess 
at X, which takes the place of the approximation Oo in the tabular 
solution just described. 

ii. Write down in column 3 the deviations of the observations 
from fflg, and weight them. 

iii. Form the squares of these deviations, wmght them, and 
enter the weighted squares in column 4. 

iv. Take the weighted averages of columns 3 and 4. Call 
these averages P and Q. They ore the correction factors to be 
used in finding x and s^, according to Eqs. 6 and 6, ahead. 

The weighted mean and standard deviation of the n observations 
are then calculated by writing 

a = Oo "H -P (®) 

s^ = Q-P^ ( 6 ) 


Now the correction factor P may be considered either as the 
average residual reckoned from the arbitrary datum oq, or as the 
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distance between x and oq. Q is the average squared residual, 
when the residuals are reckoned from oq, and s® is the average 
squared residual, when the residuals are reckoned from a. In 
words, Eqs. 6 and 6 state that , 

X = (the arbitrary datum Qq) + (tbe average residual 

reckoned from ao) (7) 

8® = the average squared residual reckoned from a 
= (the average squared residual reckoned from oq) 

— (the distance between a and oq)® (8) 

It can be seen from Eq. 4 that the extreme left entry in Row II 
(p. 150) divided by Z is 

Q-P^ 

which is none other than s\ Therefore the (minimized) sum of 
squares S is just (2 ii))a^. Hence the tabular scheme shown 
above for calculating A and S is equivalent to the steps outlined 
for the rapid method for getting S and s®. 

It is important to note that the entry £ w(.r — oq)® in Row 2 
in the “ 1 ” column arises by summing squares of deviations 
reckoned from Oq, and the quantity in Row 3 just below it is pre- 
cisely the amoimt by which £ w(x — oq)® must bo diminished to 
get the minimum sum of squares, called S, Likewise, the quantity 
Q arises by summing squares of deviations reckoned from ao» and 
P* is the amount by which this sum of squares must be diminished 
to get (see Eq. 6). 

In this example, both in the tabular solution and the " rapid 
method " for computing S and the number oo need not be cloae to 
o. It can have any value whatever, but in practice it will usually 
be a rounded-off guess at the mean (which is the finol value of a). 

60 . Systematic procedure for fo rming the normal equations for 
the parameters. There will bo a formula to be fitted. It might be 

y = a + l>x + cx^ (9) 

or it might be 

( 10 ) 


y = ae' 
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or it might be something else. Whatever it is, we can transpose 
one member and write it in the form 

Fix, y\ a, h,e) = 0 (Same as Eq. 1, p. 130) (11) 

Thus, for Eq. 9 above, F would be ^ - (a + bs + cx^), and for 
Eq. 10, F would hey — 

The reader may recall the use of the symbol Fq in the preceding 
chapter (Sec. 52). Fq stood for the value of the function F at some 
particular observed point X, Y (not a calculated point), and evalu- 
ated furthermore with the ap-proxirmte parameters Ooi bo, cq. For 
instance, for Eqs. 9 and 10, having fixed the form of the function F 
in the manner just described, the numerical value of Fq at the 
(observed) point X, Y would be F - (oq + boZ + coZ^). For 
Eq. 10, Fq would be F — 

In fitting a function by least squares, the first thing to do is to 
fix the form of the function F by transposing all terms of the for- 
mula to one side of the equation, to get it in the form of Eq. 11. 
The steps then to be followed are outlined below. It is interesting 
to compare these steps with those of Section 33, wherein there 
were no parameters. 

1st step, (o) Work out somehow satisfactory approximations 
oo, bo, Co for the parameters (cf. the reduced type in Secs. 25 and 
55) ; (b) calculate numerical values of Fo at every point. 

In some problems, depending on the formula and the weighting, 
it is permissible to take oq ■= hj = co = 0 when calculating f q 
( but not L), in which event the residuals A, B, G turn out to be 
the adjusted values —a, -b, — c themselves (cf. Exercises 4, 6, and 
10 in Sees. 65-6 ) . But this is not usually advisable even when per- 
missible. As a matter of saving time, a good rule le to commence the 
adjustment with as good approximations oo, bo, cq as con he found 
with a reasonable amount of trouble, and thus to out down the 
number of figures required in the formation and solution of the 
normal equations. 

2d step. This step requires ■ some differential calculus. It 
consists of writing down the various derivatives of F, namely 

Fa, Fb, Fo, Fa, and Py 
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The first three may be needed in forming the normal equations, the 
last two for calculating 

+ (Same aa Eq. 8, p. 134) 

Wx % 

and the summation required at the n points. L may vary from 
point to point, and some or all of the derivatives F^, Fb, and 
almost surely will. So will Fq. 

Sd step. Work out the numerical values of Fa, Fb, Fc, and L, at 
eveiy point. The following tabulation is suggested . 

TABUS 1 


Pehuminary to the matrix (3d step) 



Of course auxiliary columns may be required, depending on the 
problem and the whims of the computer. Or, perhaps some 
columns listed will not be needed, e.g., if Wx were « all the way 
down (x free from error) then E* and Wx would be omitted, since 
y alone would contribute to L, which would be merely FyFy/v>y. 
Likewise, if y were free from error all the way down, then the Fy 
and Wy columns would not be needed, for then L would be simply 
FxFx/wx. 

4A step. Divide each entry under Fa, Fb, Fe, and Fq by the 
corresponding V L. The sums at the right or bottom (one but not 
both) of Table 2 can be formed by cumulating these quotients in 
the horizontal or vertical, the individual quotients being entered in 
the table. (This cumulation requires a machine with a double 
multiplying dial, one to be locked for cumulating quotients, while 
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the other clears when desired. See a remark following Table 2 in 
Sec. 33, p. 72.) 

TABLE 2 


The mathix fob the fobmauon of the nobmal fujtjations^ (ton btbp) 



The sumfi at the right and along the bottom are used for checking 
the formations of the normal equations exactly as was done with 
Table 2 ia Section 33. First of all, the sum across the bottom 
should equal the sum down the right-hand side, as indicated by 
the check mark. In running down tiie columns, cumulating squares 
and cross-products (the fifth step, p. 156), the final total m the multi- 
plier register will equal the sum at the bottom of the multiplier 
column provided no changes in sign occur in the multiplicand 
column. In a machine with a double multiplier register, one part 
of which can be locked for cumulation while the other clears, 
individual multipliers can be checked at will in one dial, while 
the sum of the multipliers cumulates in the other one for checking 
at the bottom. 

A maximum of three or four significant figures in any column 
will suffice. Tills means that if there is great variation in the 
mzes of the numbers in any column, some entries in Table 2 may 
have only two, or one, or not even any figures; see, for instance, 
pages 213 and 224; also page 79. 

The denominations of the different columns should be made 
uniform by writing powers of 10 at the top of each row, to apply 
to the whole column (see the solved examples at the end; also the 
one in Seo. 34). No attention need be given to the powers of 10 
until the end, when the solution of the normal equations is decoded. 

* Concerning the use of the term matrix here, see the note appended to 
Table 2 in Ch. VI, p. 72. 
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Sih step. Form the normal equations from Table 2 by the famil- 
iar process of adding squares and cross-products of columns. 
Thus, no mattei' how complicated the weighting, and no matter 
what be the form of the fitted curve, the whole procedure is uni- 
form, and we are brought to a uniform and familiar process for 
the formation of the normal equations. 

As already suggested at the commencement of this section, the 
student should compare this matrix with the previous Table 2 oE 
Section 33 (p. 72), which arose in the consideration of conditions 
not containing parameters. The headings in the tables are different 
there, of course; but to the computer, the routine procedure of 
forming the normal equations from Table 2 is the same here as it was 
there. Also, the routine of solution is tire same (compare Secs. Zi 
and 61). The exercises in Chapter X will provide practice in the 
necessary steps for sotting up the normal equations for several 
-typee of functions. 

Remark. By the procedure here explained for the formation 
of Table 2, whence the normal equations are to be set up, the 
solution of the normal equations (i.e., the values of A, B, C, 
etc.) is uneguivoed. That is, it does not matter in what form 
the equation to be fitted is written. If one had, for example, 
y = os®*, he could put the same equation in the form In 2 / = In a 
+ hx, using in the fonner case F = y — oe'’* and in the latter 
case, / =lnj/ — Ina — 6 s. Further illustrations wifi occur in 
the exercises of Chapter X. When the normal equa-tions are 
made up according to the steps outlined above, the results will 
be the same frm, any form, of the fitted equation, to within higher 
powers of the residuals. 

As another example, the straight line can, be written as 3 / = a 
+ 6s or as s = — 0/6 + y/b, and F may be y — o — 6 s or 
s + a /6 — y/b. Either way, the results will be the same to 
within higher powers of the residuals. Summed up, the re- 
sults — tJie find calculated parameters, and the adjusted observa- 
tions, are independent of Ihe form in which the equation is written. 

Very large residuals, i.e,, very rough data, w^ invalidate this 
statement to some extent, but if the data are as rough as that, 
they may not be worth fitting anyhow. See some other remarks 
in Seotion 26, also in Exerdses 18 and 23 of Chapter X. 

61. Systematic solution of the normal equations. The recip- 
rocal matrix. Systematic computation of S. As has just been 
noted, the sums of squares and croBS-produots occurring in the 
normal equations for curve fitting (p, 136) ore formed directly from 
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Table 2 of the preceding section. Thus, the summation [FaFJL] 
on page 136 is the sum of squares under the column of Table 2 
headed Fo/VL; the summation {FJFi/L\ is the cumulation of 
cross-products under the columns of Table 2 headed Fa/VL and 
Fi/y/L; [FoFo/Ij] is the cumulation of cross-products under the 
columns headed Fa/ VL and FqIVL-, etc. 

We sbR.11 suppose that the normal equations have been formed 
in this manner. The numerical values of the squares and cross- 
products called for on page 166 will be entered as numbers in Eows 
1, 2, 3, 4, page 158. On this page, the abbreviated symbols 


[aa] for 

'FJ'al 
L L J 

[a5] for j 

m 

[oo] for 

[^] 


etc., have been introduced for convenience. The unit matrix in 
the columns Ci, C^, Ca is entered for the calculation of the recipro- 
cal matrix, and the sums at the r^t are formed for checking. The 
Gauss symbols [65.1], [cc.2], etc., seen in Eows II and III, will 
facilitate reference to certain entries later on, as in the exercises 
beginning on page 161. 

The solution proceeds according to the operations of multipli- 
cation and addition indicated by the directions under the column 
ViB afi fiii “ How obtained.” The procedure here outlined is similar 
to Doolittle’s^ solution, which in turn goes back to Gauss.® The 
check marks show the “ sum check ” at the pivotal points. The 
normal equations on pages 82 and 83 were solved this way. Fur- 
ther numerical examples occur in Chapter XI. Note that A is 
eliminated in Eow II ; A and B are both eliminated in Eow III. 
The values of the parameter-residuals appear in Eows 11, 12, and 
13, in the “ 1 ” qolumn. 


® M. H. Doolittle, Coast and Oeodelic Survey Report for 1878 (Washington), 

App. 8, pp. 116-118. _ _ mi. 

® Gauss, Supplementum Theoriae CombincUimiB (Q6ttingen, 1826; Werke, 

vol. 4), Art. 13. 
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Row 11 comes by dividing III through by [cc.2] to get C. 

Row 12 comes by substituting from 11 into II to get B. 

Row 13 comes by substituting from 11 and 12 into I to get A. 
This is the " back solution.” 


The nobual equations and tbeiii bolution 
VnJmemu 


Row A 


B 


C = 


Cl Ci 


Ca Sura 


[oo] 


How 


m 

Ibb] 


[ac] 

M 


[oo] 

[to] 

[co] 


0 

0 

1 


V 

V 

V 


4 

obtained 



[oo] 

0 

0 

0 

... V 

6 

IX - 
[ab]/[a<i] 

(flb]» [ac][ab] 

[oo] [oa] 

[ao}[al)l 

[aa] 

_[o^ 

[oo] 

0 

0 


II 

2 + 6 

[66.1] [to.l] 

[to.l] 

_ 

[oo] 

1 

0 

... V 

6 

IX -[oc]/[oo] 


... 

... 


. . . 

... 

7IIX- 

Ibe.ll/'Ito.l] 


... 

... 

• * t 


, . . 

III 

3 + 6+7 

[cc.2] 

[co.2] 

... 

. . . 

1 

... V 

8 

I X -(ao]/[ao] 

1 

F|I 

1 M 

Ill 

1 

0 

0 


9 

II X -[to.l]/lto.l] 

[to.l]® 


[6o.l] 

0 


[66.1] 


[66.1] 


10 

m X -[co.2]/[cc.21 

[co.2]* 

[cc.2] 

... 


lco.21 

[cc.2] 

IV 

4 + 8 + 9 + 10 

S 

... 



... V 

13 

I solved for A 

A 

Cu 

CIS 

Cl3 


12 

II solved for B 

B 

C21 

csa 

C23 


11 III solved for C 


C31 

C32 

cas 

... V 


Note. The eUipsis (• • ■) in the tabular array denotes a space wherein a 
number would ordinarily be entered in numerical calculation, but in which it 
Is not worth while to show the entry in symbols. 
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The reciprocal matrix is set off in Rows 11, 12, and 13, in the 
columns headed Ci, Ca, Ca. The entries cu, C21, csi, in the Ci 
column are the values that would be obtained for A, B, C if the 

1 

right-hand members of the normal equations were 0, as found in 

0 

the Cl column. Likewise, the entries C12, C22, C32 in the Ca column 
are the values oi A, B,C that would be obtained for A, B, C if the 

0 

right-hand members of the normal equations were 1, as found in 

0 

the C 2 column. Similar remarks hold for the entries C13, C23, Cas 
in the Ca column. The values of the elements of the reciprocal 
matrix, in terms of the coefficients comprising the normal equa- 
tions, are given in Exercise 2a, following this section. 

Remark 1. Many variations of the procedure shown on 
page 158 have been published. Each possesses merits peculiar 
to the machines available, preference of the operator, and 
other circumstances. The computer should be expected to 
develop variations that are advantageous to the peculiar re- 
quirements and conditions under which he works, and to his 
likes and dislikes. 

Remark S. Methods of solution quite different from that 
described above have been contrived, but not yet adapted to 
moss production. Some of them are devices for calculating 
the reciprocal matrix to be used as a multiplier, for example, 

T. Smith’s,^ and a very promising scheme of matrix squaring 
devised by Hotelling and Girshiok on the basis of a theorem 
regarding the characteristic equation of a determinant.^ In an- 
other direction there is Kelley and Salisbury’s® ingenious accel- 
eration of an iterative process usually known as Seidel’s (1874), 
though described earlier by Gauss and Jacobi,’' the same being 

* T. Smith, “ The calculation of determinants and their minors,” Phil. 
Mag., vol. 3, 1927: pp. 1007-9. 

® This was published by M. D. Bingham, J. Amer, Blot. Assoe., vol. 36, 
1941: pp. 530-634. 

® Tmman L. Kelley and Frank S. Salisbury, J. Amer. Sial. Assoc., vol. 21, 
1926; pp. 281-292. 

^ Whittaker and Robinson, CiAculus of Observaiions (Blaokie & Son, 1924), 
Art. 130. 
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particularly effective when good initial approximations are 
available. Then there is a fascinating pivotal process invented 
by Aitken® in .1032, after T. Smith’s method; he has now, 
however, superseded this solution by the introduction of a 
number of important unpubMied refinements. It is also 
interesting to note that electrical circuit machines, capable 
of solving something like 10 linear equations, practically instan- 
taneously after plugging the coefficients, are in operation and 
undergoing further development at several centers. 

Remark 3. The reciprocal matrix contains the variance and 
product variance coefficients for the parnmoters a, b, c. Its 
use in this connexion will be illustrated in Section 62. 

Remark 4- The reciprocal matrix has also another use, 
namely, as a multiplier for finding the unknowns in the normal 
equations, m the same manner in which it was used in Eqs. 23, 

24, and 26, on pages 93 and 94. The theory of the reciprocal 
matrix as a multiplier originated with Gauss.’* The essentials 
of this theory are contained in the exercises following. 

The " reciprocal solution,” gotten by using the reciprocal 
matrix as a multiplier, is a very sensitive indicator of instability. 

It is just for tliis reason that the reciprocal solution is lilcely 
to brealc down in the case of near indeterminacy^ — a fact 
that detracts rather di'astically from its usefulness in the solu- 
tion of normal equations in curve fitting, where near indeter- 
minacy is surprisingly common. Near indoterminney exists 
when A, the determinant of the coefficients, is very smell. 

The freezing of the solution — the near vanishing of one of the 
extreme left coefficients (such as the entry [cc. 2] in Row III) — 
is indicative of near indeterminacy, which is usually but not 
alwa 3 rs accompanied by instability. 

With regard to the source of near indetenninacy and the 
remedy, Pahner^^ gives this excellent advice. "... it occasion- 
ally happens that one of the equations is so nearly a multiple or 

® A. C. Aitken, “ On the evaluation of determinants, the formation of their 
adjugates," Proc. Ediriburgh Maih. Sac., vol. 3, 1932: pp. 207-219. 

® Gauss, Supplementim Theoriae CrmMruiiionie Erroribus Minimis Obrnmae 
(Ghttingen, 1826; Werks, vol. 4), Art. 8. 

“ See a paper by the author in Science, May 7, 1937; also Henry Schulte, 
The Theory and Measurement ttf Demand (Chicago, 1938), pp. 761-3. Ap- 
pendix C can be highly recommended for techniques of curve fitting. 

A. de Forest Palmer, The Theory o/ Measurements (McGraw-Hill, 1912), 
p. 77. This book, by the way, is one of the best on experimental science and 
scientific inference. 
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submultiple of anothOT that an exact solution becomes difficult 
if not impossible* In such cases the number of observation 
equations may be increased by making additional measurements 
on quantities that can be represented by Icnowu functions of the 
desired unknowns . The conditions under which these measure- 
ments are made can generally be so chosen that the new set of 
normal equations, derived from all of the observation equa- 
tions now available, will be so distinctly independent that the 
solution can be carried out without difficulty to the required 
degree of precision.” 

Remark B. An important consideration in the solution 
of equations is the maximum error in the values found for the 
unknowns — maximum error, not just the average or standard 
error — arising from errors in the coefficients. Tuckerman^* 
shows a simple procedure by which this maximum error can be 
determined. 


Exercibeb 

Exercise 1. (a). The determinant of the coefficients of the 
normal equations on page 168 can be evaluated as 

[aa] [ab] [oc] 

A s [ab] [bb] [be] = [aa] [bb.l] [cc.2] (12) 

[ac] [be] [cc] 

which is to say that the determinant A is the product of the extreme 
left numbers in the Roman-numbered Rows I, If, III. This result 
is important, because it shows that in near indeterminacy, i.e., 
when A is small, one of these factors on the right will be small. 
The so-called phenomenon of freezing (the vanishing of [bb.l] or 
[cc.2]) is thus associated with a small determinant, which usually 
but not always gives rise to instability. (See the reference to 
Tuckerman below.) 

Hint; Ohio’s pivotal expansion will be found admirably 
suited to the demonstration of Bq. 12. The work might pro- 

B. Tuckerman, Annale of MaOu SUUislics, vol. 12, 1941; pp. 307-316. 



162 CONDITIONS CONTAINING PARAMETERS [Sec. 61] 

ceed as follows, the pivot element being unity in the unner 
left corner. 


A s 


[aa] [ah\ [cue] 

[oh] [bb] [6c] 

[fflc] [6c] [cc] 


= [oo] 


^ [a6] [oc] 

[ofl] [aa] 
[a6] [66] [6c] 

[ac] |[6c] [cc] 


= [oa] 


= [aa] 


M-Mg M-Hg 

[i&.l] [5c.l] 

[ic.l] M-[ac]|^, 

[aa] 


[ao] [66.1] 


1 M 
[ 66 . 1 ] 

[6c.l] [cc]-[ac][^ 

M 


= [oo] [66,1] [cc.2] 

(6) Show that none of the extreme left entries in Rows I, II 
and III can be negative. {Hint: Make use of Sec. 29 . Or, use the 
Schwarz-Chnstoffel inequality.) 

Exercise The matrix reciprocal to A can be denoted by 


A~^ s 


CU Cl2 Ci3 
^21 C 22 C 23 

Csi C 32 C 33 


( 13 ) 


coSanS of the normal equations with the 

constant columns Ci, Cs, and C3 leads to the values 


c,, = E2L°fW, 

A 


cof. of [ob] 
C12 —i—l, 

A 

A 


Cl3 = 


cof. of [ac] 


cof. of [6c] 

C 23 = — 
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cof. of [ac] C(rf. of [ 6 c] cof. of fed 

C31 1 ' C32 = C 33 ^ 

The abbreviation cof. denotes cofactor. 

Since A falls in the denominator of each element, a Htnall value of 
A, near indeterminacy, results in high standard errors of the 
parameters; see Exercises 2a and 12b of Chapter X. 

(6) Like the determinant A of the coefficients, the reciprocal 
matrix A“^ is symmetrical, i.e., C12 = C21, C13 = csj, C23 = 032. 

(c) The matrices A and A“^ are also alike in another respect — 
the terms on the main diagonal -will always be positive. 

(d) Show that C33 = l/[cc.2] = the reciprocal of the coefficient 
of the third unknown in Kow III. 


Exercise S. (a) Combine Eq. 17 on page 57, and Eqs. 10 on 
page 136 to get 

S = [00] - [aojii - [ho]B - [co]C (14) 

Show also that 

[ao] [ab] [ac] [ao] 

[ob] [6b] [6c] [bo] 

[ac] [6c] [cc] [co] 

[ao] [60] [co] [00] 

S (16) 

[ao] [ab] [oc] 

[ab] [hb] [be] 

[ac] [be] [cc] 

Hirtt; Expand the numerator and get 


S = — [ao] 


[ab] 

[ac] 

[oo] 

[aa] 

[ac] 

[oo] 

[ 65 ] 

[6c] 

[60] 

[06] 

[6c] 

[60] 

[6c] 

[cc] 


[oc] 

N 

[CO] 


[aa] [ab] [ao] 

[06] [66] [60] 

[ac] [6c] [co] , , , A 

— [coj 1- [00] — 


A 
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which reduces to Eq. 14 when it is observed that the coefficient 
of — [flo] is none other than A, the coefficient of [6o] is —B, and 
the coefficient of — [co] is C. 

(b) Prove that the extreme left entry in Row IV of the solu- 
tion exhibited on page 158 is actually S. Thus, the minimized 
sum of squares of the residuals conics automatically in the routine 
of the solution. 

(c) Show alsOj by noting how Rows 8, 9, and 10 are formed, that 

S = [ 00 ] - - [bb.\]B'^ - [cc.2]C=> (16) 

where A" = [ao]/[afl] = the value of A that would be obtained if I 
and c were fixed (not adjustable) at the values bo and co, 
and wherein also = [bo.l]/[bb.l] = the value of B that would be 
obtained if c were fixed at the value co, but a and b both adjustable. 

Remark 1 . This result sheds a singular elegance on the form 
of the solution exhibited on page 158, The term [oo] seen in 
Row 4 is the sum of the weighted squares of the residuals calcu- 
lated under the assumption that a = ao, b = bo, c = co (see 
Exercise 3 of Sec. 58, p. 145). The three negative terms in the 
“ 1 ” column of Rows 8, 9, and 10 on page 158 are precisely 
the amounts subtracted frora[oo]bythe terms on the right of Eq. 

16, and in the same order. That is to say, by the routine solu- 
tion outlined on page 168 there will appear (1") in Row 8 the 
reduction in weighted squares that is brought about by allowing 
a to be adjustable while b and c aro fixed at bo and Co; (2°) in 
Row 9 the further reduction that is accomplished by allowing 6 
to be adjustable while e is held at co; and (3°) in Row 10 the 
final reduction that comes from allowing c to be adjustable, the 
net result being the minimized sum of weighted squares, S, in 
Row IV. 

After a solution has been carried out upon the parameters a, 
b, e, the question often arises, what would have been the result 
for S if the parameter c had not been adjusted, but had been 
fixed at (say) 70 ? Now if this 70 is hot too far from the final 
value of 0 , one need only add [cc.2] (c — 70 )* to S in order to see 
what vxnM have been obtain^ for S had 0 been fixed at 70 (see 
Examples 1 and 2 of Chapter XI). The value of u* (fixt) would 
then be S + [cc,2](c — 70 )* divided by 71 - 2, not n-3 (ft = 
the number of points). 

Tinder certain conditions, the restriction that 70 and c be not 
far apart can be removed; the polynomial j/ = a + ba: + ex* 
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with X free of error is an example. It all depends on whether 
the parameter c enters the L factors of Tables 1 and 2 in Section 
60 . If it does not, then no matter how wide the disparity be- 
tween c and 70, the term [efl.2] (c — 70)* still represents the incre- 
ment in 8 that would be brought about by adjusting a and 6 to 
the condition c = 70. 

In like manner, and under similar restrictions, a term 
(6 - PoY/oii wiU represent the increment in S that would be 
brought about by a^usting a and c to the condition & = /3o 
(see Exercise 2 d). 

Similarly, the two terms [ 66 . 1 ] (6 — do)® and [cc. 2 ] (c — 70)* 
added to the 8 found in Row IV will give what would have been 
obtained for 8 if only a had been adjusted, 6 and c fivAd at do 
and 7o. In this circumstance, tr^exl) would be computed with 
ji — 1 degrees of freedom. 

It is important, as a practical matter, to note that the coeffi- 
cients [66.1] and [cc.2], needed for these increments, are already 
at hand, numerically, in Rows 11 and III in the finished solution, 
page 158 . 

Remark £. In both parts (o) and (c), 5 is shown as three 
terms subtracted from [00]. Evidently 

[ao]A -I- [6o]B -h [co]C = [aa]A"* -h [ 66 . 1 ]B'® -|- [cc. 2 ]C* 

(d) From Eq. 16 it follows that if c be changed by the amount 
ic, while a and 6 remain fixed, the change SS in the sum of squares 
obeys the relation 

^ / Sc y ( 17 ) 

\S.E.ofc/ 

Exercise 4. Prove that the solution for A, B, and C found from 
the “ 1 ” column will also be given by the equations 

A = [ao]cii -(- [bojcis + [cojcis 
B = [oo]C21 + [6 o]C22 + [co]C28 ( 18 ) 

C = [flo]cai -I- [bo]cs2 + [cojcas 

This method of finding the unknowns A, B, and C is called the 
reciprocal solution because the reciprocal matrix is used as a multi- 
plier along with the constant (“ 1 ”) column [ao], [60], [co]. The 
reciprocal solution is particularly useful when the same coeffi- 
cients, hence the same reciprocal matrix, are repeated over and 
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over from one problem to another, but with a new constant col- 
umn for each problem, and hence with a new set of values for A 
B, and C each time. See, however, the reference to difficulties 
that may be encountered in near indeteiminacy, mentioned earlier 
in this section, also in Example 1 of Chapter XI. Theoretically, 
the direct and reciprocal solutions should agree, and they will if 
the computer carries enough decimals. 


In matrix notation, the results of tliis exercise can be ex- 
pressed as 

Av = H 

where A is the matrix of the coefficients of the unknowns. 

[aoJl 

H is the matrix of the “ 1 ” column, namely, [6o] 

[co] 

and A 

V is the matrix of the three unknowns, namely, B 

C 

The solution of the above equation is 
« = A-i H 

To evaluate A“^ we set 


and find 


A c = 1 (the unit matrix) 
c = A“‘ 


Having now the matrix A~i, we use it as a multiplier with H to 
find the matrix v from the relation above, getting v = cH. This 
is the matrix expression for the results stated in the preceding 
exercise. For illustrations of the reciprocal solution see Sec- 
tion 36 and Examples 1 and 2 of Chapter XI. 


Exercise 5. Prove that the values of the determinants A and A“^ 
are reciprocals. (A and A“* defined on page 162.) 

Exercise 6. (a) If a: = r cos 6, y = r sin 8, the two Jacobiana 
as matrices, namely. 


dx 



dr 

de 

dr 

dr 


dx 

dx 



and 



dx 



dr 

dB 

dd 

de 


dy 

dy 
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are reciprocals of one another; i.e., their product gives the unit 
matrix 

1 0 
0 1 

(b) Show that a similar relation exists in three dimensions. 

In the derivatives taken with the symbol d, d is constant 
while X, y, and r vary, and again r is constant while x, y, and d 
vary. In the derivatives taken with the symbol d, a: is constant 
while r, 6, and y vary, and again y is constant while t, 6, and 
a: vaiy. 

62. The weights of the parameters; their standard errors. 
The standard error of a function of the parameters. The standard 
error of a curve. It is a fact^® that the reciprocals of the weights 
of the parameters are found on the diagonal of (see Exercise 2 
of the preceding section), i.e., 

1 1 1 /inN 

Wa ' Wb > We = — (19) 

Cu C 22 C 33 

where 

cjj s var. coeff. of a, C 22 = var. coeff. of b, C 33 = var. coeff. of c 
Then, since weights are reciprocalB of variance coefficients (p. 21), 


Clltr^, ffj® = C22ff^ Vfl* = CsbO^ 

(20) 

(S. E. of a)2 = Ciia^ ] 


( •' “ b)^ = C22tr® 1 

(21) 

( “ " c)2 = C38<r* J 



Let / he a function of the parameters. Then 

= (/flVa)^ “|“ 2(/a/brabO'o<rs fajc'f'ao^a^e) 

+ {fb<yb? + 2/t/or6cV6<ro + Cfoffo)^ (By Eq. 7, p. 40) 
= + 2C12/0/6 + 2 cis/o/o + C22/4* 

+ 2C23/i,/c + C 33 / 0 *) (22) 

The theory of all this goes back to Gauss, Theoria Combinatimis, Art. 21. 
An excellent referenoe is Whittaker and Robinson's Calculus tjf Obseniaticms, 
Arts. 121-123. 
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As in Section 13 we write for the unbiased estimate of by 
external consistency, 

g 

o^{ext) = —1 where k = n~ -p 
k 

n being the number of points and p the number of adjustable 
parameters. When a is not Icnown from any better source, tliis 
estimate may have to suffice, and would replace o-® m Eqs. 
21 and 22, giving respectively 

(Eat’d S. E. of a)® = cn <r^{ext) 

( « « « b)2 = C22(r^(fti:i) , ■ (23) 

( “ “ “ Cf = C3ia\(S3it) 

and 

(Est'd S. E. QiSf = + 2ci2/a/!. + 

+ C22/b® + 2c23/b/c + Css/c^l (24) 

The student is urged to study Chapter V of R. A. M-sher’s 
Statislical Mdhods far Research Workers, wheroiu examples of 
the manipulation of the reciprocal matrix will be found. 

63. The error bands associated with a curve. Rejection of 
observations. When we write 

y = fix; a, b, c) (25) 

and ask for the standard error of y, wo are merely asking for the 
standard error of a function of a, b, and c, but not of x; conse- 
quently, we can apply Eqs. 22 or 24 at once, z enters merely as 
a constant. 

The distinctioa between Eqs. 22 and 24 is that, in the former, 
ff is supposed to be known or approximated closely enough, under 
conditions of randomness, as from previous experience, or from 
internal consistency (Sec. 13), or from any other source that does 
not depend on the way the particular points in question fit the 
curve. In Eq. 24, on the other hand, a is estimated from the fit 
of the points, as was explained in Section 13. Eq. 22, when applied 
at abscissa z along the fitted curve, gives the standard error of the 
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curve for that particular abscissa. If this calculation is Tna.rlpi for 
several abscissas, one may plot points along the standard error 
band. Error bands are plotted to show one or more standard 
errors above and below the fitted curve. In Fig. 22 (p. 228) the 
band shown is d= 1 .96 standard errors. This width of band would, 
on the average, leave 5 percent of the points outside the band if 
the coordinates were distributed normally about their true values 
with standard errors afs/wg for x and o-/\/«)j, for y. Sometimes 
the “ probable error ” band is drawn, or a multiple thereof. 

When (T is estimated from the fit of the points and used in Eq. 24, 
one obtains a confidence band for the curve. A confidence band 
is different in principle from an error band only in using a (ext) in 
place of a presumably better Imown value of a. This, however, is 
often an important difference, because, although tr itself is sup- 
posed to bo constant, the external estimate of v will vary from one 
experiment to another, even in randomness, unless n — p is as 
large as 20 or preferably 30. The width of the band may be 
adjusted to give various degrees of “ confidence.” This is done 
by using an integral of Student’s distribution, which is easy to do 
by looking up the corresponding value of t in Fisher’s tables^® for 
n - p degrees of freedom. Thus, to compute a 95 percent con- 
fidence band, one would look up foe Ia Fisher’s table, and then 
compute \ Y — y\ for several values of x, using the equation 


jy-yl 

Est’d S. E. of y 


(26) 


A probable eiTor, for normally distributed observations, is 0.67 times the 
standard error. For other distributions, some other factor is required, but 
calculations are ordinarily made with the factor 0.67 for which ■§■ is a dose 
enough approximation. Birge uses the probable erro; band along with his 
curves. The following papers of his are recommended for their scientific 
insight, and for simple derivations of the standard error formulas: (o) Physical 
Rmew, vol. 40, 1932: pp. 207-261; (b) Amer. Physics Teacher, vol. 7, 1939: 
pp. 361-367. 

V. A. Nekrassoff’s very handy nomograph may be used. It was pub- 
lished in Melron, vol. 8, No. 3, 1930, and is reproduced in W. A. Shewhart’s 
Economic Control of Quality (Van Nostrand, 1031), p. 490; see also Demlng 
and Birge Slaiistical Theory of Errors (The Graduate School, Department of 
Agriculture, Washington, 1938), p. 136. 
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The distance \Y — y\ laid off above and below the curve defines 
points on the confidence band, and the result will have the appea^ 
ance of Fig. 20. (The capital Y used here is not to be confused 
with the same letter used in Fig. 17 and elsewhere for an observed 
coordinate.) 


Y 



Pio. 20. A fitted curve oad the corresponding confidence bond. An error 
bond is laid oS in like manner as a multiple of the etnadord error of the func.* 
tion, and has a similar appearance. 

Remark 1 . A convenient reference showing the application of 
Eq. 24 to curve fitting is a paper by Heniy Schultz, J. Amer, 

Stat. Assoc., vol. 26, 1930: pp. 139-185. Schultz shows curves 
and confidence bands of width twice the standard error of the 
curve for sevei'al kinds of curves. It should be mentioned, as 
Schultz does, that all these things were well known to Gauss and 
others in his time, but that they did not talce the trouble to write 
out the formulae explicitly and draw the graphs for all the things 
that interest us today. 

Remark 2. It must be remembered that, even in a state of 
randomness, a new set of points (i.e., a new experiment) will give 
a new curve and a new set of parameters; hence, curve and error 
band, will be shifted to a new position by a new experiment. 
Moreover, since the external estimate of <r will fluctuate from 
one set of data to another, then not only will the curve and con- 
fidence band be shifted to a new position by a new set of data, 
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but the width of the band itself will also be different. This is 
one of the reasons why a single experiment, without considerar 
tion of other knowledge, is not a basis for action, particularly if 
the consequences of the wrong action are hazardous (of. Ch. I). 

Confidence intervals for any other function of a, 6, and c are 
made up in like manner, and similar remarks apply. 

The purpose in drawing an error band or confidence band is to 
invoke statistical aid in detecting spurious conditions in the data, 
or more precisely, in the experimental conditions that gave rise to 
the data. A point that lies outside an error band of width two or 
three standard errors should be investigated; but it is to be dis- 
carded, and the curve refitted, only if investigation discloses 
anomalous experimental conditions at that point. Whether one 
uses a band of width two standard errors or three standard errors 
is a matter that can be decided only by personal preference and 
experience in a particular line of work. The wider the band, the 
fewer the points outside it, and on this criterion the leas likely one 
is to look for experimental difficulties. On the other hand, if the 
band is too narrow, one will look for experimental difficulties too 
often — that is, ho will be looking for trouble too often when there 
is no trouble.^® Many papers and chapters have been written on 
the statistical rejection of observations, but the best practice seems 
to be contained in the statements just given. In summaiy, a 
point is mer to be excluded on statistical grounds alone}'’ 

“ These thoughts follow the reoaoning expounded by Shewhart in 1924 
when he introduced the control chart. The student of modern statistical 
theory will recognize in them the arguments inherent in errors of the first and 

second kinds. , 

R. A. Fisher, " On the mathematical foundations of theoretical statistics, 

PM. Tram. Royal iSoc., vol. 222A, 1922: p. 322 in paxtioulai. 
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CHAPTER X 

EXERCISES ON FITTING VARIOUS FUNCTIONS 

64. Purpose of the chapter. The exercises and notes in this 
chapter serve two purposes: first, to provide practice in 
forming the normal equations for various functions commonly 
met in practice; second, to provide a compendium of results, 
handy for reference. Once these exercises are mastered, other 
functions that arise in practice should present little or no difficulty. 

A special note should be made concerning the fitting of poly- 
nomials such as 

y = a + hx, 2/»a-hba:4- cx\ etc. 

When a: is free of error and uniformly spaced, certain short-cuts, 
eminently worth while learning if the problem is to occur fre- 
quently, are provided by the use of orthogonal functions. Since 
good references are accessible, the subject need not be treated 
here. The methods shown in the following exercises will work 
under very general conditions. But if a polynomial is to be 
fitted again and again when x is fm of error and equally spaced, 
the reader is advised to learn the method of orthogonal func- 
tions. The theory is complicated, but the application is not. 

The Mowing list of references will suffice for clear descriptions 
of several different procedures; 

1. R. A. Fisher, Staiistied Methods for Research Worhm 
(Oliver & Boyd); sections 28, 28.1, and 29.2 in the 6th and 
later editions. iWer’s procedure and his description thereof 
have justifiably found great favor. 

2. R. A. Fisher and Frank Yates, Statistical Tables for Bio- 
logical, Agricultural, and Medical Research (Oliver & Boyd, 
1038). An extension of these tables has recently been pub- 
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lished by B- L- Anderson and E. E. Houseman, “ Tables of 
orthogonal polynomial values extended to n = 104 ” (Ames, 

Research BvUetin 291, 

3. Raymond T. Birge and John D. Shea, A rapid method 
of o'aloulating the least squares solution of a potynomial of 
any degree ” {Unwmily of Cdifomia Publications in Mcake- 
madcs, vol. 2, No. 5, 1927; now unfortunately out of print). 
This procedure is rapid and possesses great merit. Up to a 
certain stage it seems to be equivalent to Harold T. Davis’ 
method, but beyond that stage the remaining work is simpler 
than Davis’, and requires fewer decimals. 

4. A. C. Aitken, Proc. Royal Soc. (Edinburgh), vol. 63, 1932- 

1933; pp. 54-78. „ . . /™ r, i 

5. Max Sasuly, Trend Analysis of Statialics (The Brook- 

mgs Institution, Washington, 1934). 
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66. Theline^ 

In the exercises that followi the B 3 nnbols [s], [a;®], \ry], [jd<'o], 
and the like, refer to summations formed with the observed co- 
ordinates. Moreover, x and y refer to the mean values of the 
observed coordinates. The distinction made in Chapters IV, 

VII, and IX — capital letters for observed coordinates, and 
small letters for the adjusted coordinates — can now be dropped. 

In this chapter and the next, it will be convenient to use capital 
letters to denote logarithms (7 for log y, etc.). When there 
seems to be special need of distinguishing observed from calcu- 
lated coordinates, the subscript obs or calc will be affixed. In 
the numerical evaluation of the derivatives, and of W or L 
(Tables 1 and 2 of Sec. 60), if x and y are called for, their ob- 
served values are to be inserted, along with the approximate 
determinations aoi bo, cq for the parameters. 

Exercise 1. (a) Given the line 

y = a -^-bx 

to be fitted to n points, x free of error, all y coordinates of equal 
wei^t (unity). Here we take 

F = y ~ (o + bi) 

The line y = bx, forced to pass through the origin (i.e., with o = 0), 
was discussed to some extent in Section 16. 
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The derivatives are 

Fa ~ -1| Pb = -af 
Fx = —b (not needed here), Fj, = 1 
L = 1 (Why? See Eq. 8, p. 134.) 

With the approximate values oo and &o we compute 
Fo = Voba ~ (Oo H~ box) 

at every point. Since L = 1 at every point, Tables 1 and 2 of 
Section 60 coalesce, and the normal equations are seen to be these; 


Row A 5=1 Cl Cj Sum 


I 


w 

-[P<A 

2 


[aa] 

-[®fo] 

3 



(W 


(Set 1, Exer- 
cise 1) 


(b) The solution for A and B, found by the routine of Section 61 
or any other method of solution, is 


B = - 


i®Fo] + m 


nm = [®®] — nJ® 


nno is the second moment of the x coordinates about an axis paral- 
lel to Oy and passing through the centroid $, 5. 

(c) The adjusted values of a and b turn out to be 

0 = 00 — A = y — b£ 


b = bo - B = £ (g - x)iy - y) ^ [xy] - nay 
W/l2 A/t2 


The fitted line therefore passes throu^ the centroid x, y. But 
note that when there is error in both x and y coordinates at some 
or all of the observed points, the weights being such that Wb/wk is 
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not constant throughout, the line does not pass through the cen- 
troid (see Remark 2 in Exercise 4). 

{d) The solution just found for a and h is the same as would have 
been found from the normal equations shown below as Set 2, in 
which the unknowns are the full values of a and b. In this prob- 
lem it is therefore permissible to take oq and bo both as zero, where- 
upon Fa is simply yoba- 


Row 0 

6 = 

1 

Cl 

Ci Sum 


I n 


[v\ 

1 

0 

(Set 2, 
Exeroiee 1) 

2 

3 

[xx] 

lw\ 

0 

0 

1 

0 


Note that a calculation of Fo is required at every pomt in 
forming the normal equations of Set 1, but not for the formation 
of Set 2, because in the latter, Fo is the same as ya,,. However, 
in Set 1 it is only the residuals A and B that are to be solved 
for, the main part of the adjustment having already been 
allowed for in fixing the approxunate values oo and 6o. It is 
different in Set 2; there the unknowns are the full values of a 
and 6, requiring the computer to carry more figures. These 
additional figures usually more than offset the time required 
for computing Fq. It therefore is usually advisable to find 
good approximations and use Set 1. The better the approxi- 
mations, the fewer figures required. Birge and Shea make 
use of this principle in their method of fitting polynomials 
(mentioned in the preceding section). 

(e) When the solution of either Set 1 or 2 is carried out accord- 
ing to the scheme of calculation exhibited on page 158, the extreme 
left entry in Row III will be the minimized sum of squares S, or 
L (yobi — Vcaie)^- The sum of squares removed from [FoFq] in 
Set 1, and from [j/y] in Set 2, by the successive adjustments of a 
and b, appear in the extreme left entries of Rows 6 and 6. Show 
that 

<S = [yy] - n’f - nnob^ 

the last term being the sum of squares removed by allowing the 
line to have slope b instead of slope 0 — in other words, the sum of 
squares remove by regression. The two terms np^ and 
appear in Rows 5 and 6 of the solution of Set 2. 
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[Sbc. 6S] 

(f) If 7 denotes yon - Vcau at any point, the solution fot a 
and b renders 2^ 7 = 0. (But note that neither L 7 nor 2 wV 
is necessarily zero in least squares solutions; it only happens to be 
so here. In fact, in Sec. 15a we saw a simple example wherein 
neither 2^ 7 nor 2 wV was zero. See Remark 4 in Exercise 4; 
see also Exercise 5.) 

Exercise S. (a) The reciprocal matrix for the normal equations 
in the preceding exercise appears in the Ci and columns of ^ 
Rows 7 and 8 (these numbers refer to the solution of the equations 
solved according to the form on p. 158). It turns out to be 

n n/ji^ nuz 

— 1 
n./x2 n/ia 

(b) From the upper left and lower right elements of this array 
we may say that 

71 

The weight of a = — — s-r~ 

The weight of b = n/ta 

Thus, if the experimental conditions were random, our confidence 
in b would increase as the " spread ” of the points increases. Is 
this reasonable? Why docs the weight of a depend on x? 

(c) (S. E. of o)2 = <72(1/71 + x^/njiiz) 

(S. E. of b)2 = ,r^/nM 2 


Note that the weights and standard eiTors of a and 6 do not involve 
the y coordinates of the points. Comparei with part d of the 
next exercise. Note also that the denominator of the last fraction 


is equal to A, since 


n^iMi = 


n [x] 
[x] [xx] 


hence near indeterminacy (a small value of A) is closely associated 
with large standard errors of a and b, and a rapid “ fanning out ” 
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of the standard error of yeau each aide of x, y (see the next part; 
also Exercise 2a of Sec. 61). 

(d) From Eq. 8, page 40, and the reciprocal matrix of part (o) 
of this exercise, prove that the 

(S. = 

w I M2 , 

Thus the standard error of yeaic is least at the center of gravity 
(S, 5) of the points, and fans out each side of it. (See Sec. 63 and 
the reference to Henry Schultz; also Figs. 20 and 22.) 

(e) The standard error of the calculated line of Exercise 1 at 
the center of gravity is ff/y/n, as it would be for n observations 
made on a single unknown. (Do this in two ways: 1° put a = 2 
in part d ; 2° put x = 0 in part c for the.Btandard error of a.) 

Exercise 3. (a) Carry out the solution of the normal equations 
of Exercise la in symbols, following the outline given in Section 61, 
and show that the minimized value of iS or of 2 (yob$ — Veau)^ 
comes in the “ 1 ” column of Row III (which will be the extreme 
left entry in III). The same is true if the approximations oq and 
lo are used, as is advised in Exercise Id. j 

(6) Show that the minimized sum of squares in this problem * 
can bo written 

£ res^ = n(l — r^)sy^ 
where 

p = 22 — — ^ = 'tliB correlation coeflScient 
and 

= [yy] - nf 

nsx^ = [axe] — nx® 


(sa^ is here used in place of M 2 for con^tency with Sy.) 
(c) The estimate of v made from the fit of the line is 


ir®(ext) 


n(l - r®)s„® 
n — 2 


(See Sec. 13) 
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id) The 

(Est’d S. E. of o)* = Sj,® ( 1 ^ — 2 J 

7h a \ 5 # / 

1 2 2 

(Est’d S. E. of 6)2 ^2 

Note that the estimated standard errors of a and b involve the y 
coordinates; compare with part (c) of the preceding exercise. 

(e) The 

1 - r® 

(Est’d S. E. of ycaic)^ 

in — i 

Exercise J^. (a) If both x and y coordinates are subject to error 
with varying precisions at some or all of the n points, one must 
perform the calculations called for in Tables 1 and 2 of Section 60. 
For the line 

y = a + bx 

one may take 

F = y — a — bx 

Some good approximate values Oo and 6o having been found, one 
can then calculate the numerical value of 

Po = Vobi ~ (<*0 "b boXob,) 
at each of the ti points. The derivatives of F are 

Fa = -b, Fy =1, Fa= -1, Ft = -X 
whereupon 





L varies from point to point with to* or Wy. 

The headings for Table 1 of Section 60 would be these : 


h, or Point No. tu* Wy L i^L Ft = —x Fo 



(It is not necessary to tabulate Fa, Fy, and Fa since they remain 
constant from point to point.) 
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The headings for Table 2 of Section 60 would be as shown below, 
ftjOrPointNo. Pa/Vi = -l/VL FhHL=-xHL Fulfil Sum 

It has already been remarked (Remark 3, Sec. 54) that there 
is some theoretical advantage in writing W in place of 1/L, 
though it is a fact that with machines having automatic division 
and two dials for quotients — one for the individual quotients 
needed for Table 2, and another for cumulating the quotients 
across the rows for the “ Sum ” column of Table 2 — there may 
be a practical advantage in tabulating V L rather than -^Wia 
Table 1, and using divisions by V rather than multiplications 
by V W, to form Table 2. 


Writing now 


W 



Wx Wy 


for L 


we see that W = lOj, if a: is free of error or if b = 0 (see the next 
exercise), and W = tOi/b^ if y is free of error, but that loth term 
are required if x and y are both subject to varying errors, and if the 
line is inclined so that 6 is not small (see Exercise 86). 

In terms of W the headings of Table 2 might be these: 


h, or Point No. 

^|W 

—xijW 

■4W-Fi 

Sum 


(5) The normal equations are formed in the usual way by sum- 
ming squares and cross-products from Table 2. They can be 
symbolized as shown in Set 1. 


Row 

A 

B 

= 1 

Cl 

Cl Sum 


I 

m 

1W®1 

-[FPol 

1 

0 

(Set 1, 
Exercise 4) 

2 


[PPra] 

-[PTrcFol 

0 

1 

3 



[WFoFo] 

0 

0 


The systematic solution of the normal equations (shown on 
p. 168) gives A and B from the “ 1 ” column, and the reciprocal 
matrix tT^ as usual from the Ci and C 2 columns. The adjusted 
values of a and 6 will be 


a = Oq — -A 
6 = bo — B 
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The systematic solution gives the minimized value of £ (WsFj* -|- 
w^Vj^) in Row III, column " 1.” That portion of the sum of the 
weighted squares subtracted from by shifting the first 

parameter from oo to a appeara in Row 5, and tho portion further 
removed by shifting the second pai-amctcr from ho to i appears in 
Row 6 (?ee Exercise 3 of Sec. 61; al.so Exci-cisc 1 of this section). 

Note that Oo aiid be taken as 0 (with the necessary 

inorease in the number of decimals required in the normal equa- 
tions), BO far as Fa ia concerned, in which event Fa becomes simply 

Fa — Vobt 

The normal equations are then symbolized like those following, 


Row a 

h 

= 1 

Cl 

Ca Sum 


I m 

[Wx] 

+[11^J/1 

1 

0 

(Set 2, 
Exercise 4) 

2 

3 

[Wax] 

MWxy] 

[Wyy] 

0 

0 

1 

0 ... 


and the solution gives a and b directly. Why aro more decimals 
required in these equations than in the preceding ones giving the 
(supposedly small) residuals A and B? 

But note carefully that an approximate value of b rrmst be used 
in the calculation of W at each point where x is subject to error, 
bo may be called 0 in the calculation of Fa (as noted above) but 
not in the calculation of W. If this admonition is disregarded, the 
effect of the weighting of a: is lost. In fact, if it turns out that the 
approximate value of b used for calculating W was too far removed 
from the final b, it may be desirable to make a second adjustment io 
secure improved weightings W, which can be obtaiaed by using the 
value of b from the first adjustment; but this is seldom found 
necessary in practice.® 

* As Gauss put it, in a somewhat different problem: " Quodsi dein oaloulo 
absolute contra exspectationem valores incognitarum p', q', t\ s', etc., tanti 
emergerent, ut parum tutum vidcatur, quadrata productaque neglexisse, 
eiusdem operationis repetitio (aceoptis loco ipsarum tt, x, p, o-, etc., voloribus 
correotis ipsarum p, q, r, s, etc.) remedium promtum afferet." Thmia Motus 
Corporum Coeleaiium (Hamburg, 1800), Art. 180. 
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Hemark 1 . Of course, it may happen insomc ptirtieiilar prob- 
, Tih akuallv is veiV small, and that 0 is therefore a pood 

J in «..l. drcum-tocc lin» i« pr^*- 

i hortontsl, the miehting of » (looi not miitlcr muih^ imd 
the^oomputer may as well simplify matters and set 11 - 7i„, 
■ vveiiditing of a: -not because .r is free of error 

not because Wz is infinite), but because h is zero or nearly 
80 The student should ponder over the situation where h is 
actually known to bo 0; do the values of x coinit at all in the 
soMion? Does this not take us back to the siinplest problem 
£“e fitting, seen in Section 10? The solution obi aincd 
there can be translated to the needs of the present circum- 
stances by interohanging x and V, and rewriting Itq. 10 on 
page 10 to get 

2- w?/ 

(L “ ^ 

£ W 

then rewriting Eqs. 12 and 12' on page 21 to get 

lOo = S’" 

and the 


(S. E. of a)® = 




In all these equations, w now denotes tho weight of y, not x. 

Remark S, When x and y ore both Hubjciit to error at sonic 
or all of the observed points, the line docs not pass through the 
center of gravity 

_ _ 

® “ W ’ ^ M 

But the line will in any oaae pasH tlirough a quad center defined 
as 

® “ [in ’ ^ [W] 


Remark S. With 1/W written in place of L, Eq. 8, page 134, 
gives 

1 = Ms + EsEs. (Cf. Remark 3, p. 135.) 

W Wx Wy 

As has already been seen, the first term drops out it x is tree of 
error, and the second term drops out if y is free of error, To 
TTmVa the change from a solution in which x is froo of error to 
one wherein both coordinates are subject to error, we merely 
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add ihe other term in 1/F and recalculate W at every point, 
the procedure being otherwise the same. There is a close 
analogy with celestial mechanioe; when one wishes to com- 
pute the orbit of a body of mass in about another of mass M, 
he may at first make the simplifying assumption that ilf is 
infinite (i.e., immovable), and solve the equations, later replao- 
ing by p where 

1 = L + 1 

PL m M 

This replacement yields the absolute motion of the two bodies, 
neither being of infinite mass (i.e., neither one immovable). 

Remark 4- When r and y are both subject to error at some 
or all of the points, we can not always assert that 

2 Vj = 0, or 2 F# = 0, or £ (WiF* + %F,) = 0 

though these may sometimes happen, os in Exercises 1 and 5, 
q.v. We have alieady seen a simple example in Section 16, 
where these summations were not zero. There is, however, a 
property of least squares by which one can always assert that 
after the adjustment,’ 

r(i«xtf,F, + %i7,F,) = 0 

(For definitions of Ux, oto., see Pigs. 16 and 17 on pp. 132 and 
138.) 

Remark 6. It is interesting to note that in the routine solu- 
tion of Set 2, the minimized 8 appears in the extreme left entry 
of Bow III, but that, in contrast with Set 1, unless the final 
value of 6 is actually or very near 0, the entry in Row 6 directly 
above 8 will wt show the increment in 8 that would result from 
fixing b at either the value 0 or &o. The reason is that a good 
value of 6 must be used in TF at each point where x is subject to 
error; if we want to Icnow what the solution would have been 
with 5 = 0, we must actually make a solution with 6 set equal 
to 0 in the computation of IF, in which circumstance W reduces 
to U)f, as already noted. 

Exercise B, Given 

p = 0 + ba: 

to be fitted to n points, x free of error, the y coordinates each hav- 
ing weight iCy, varying from point to point. This is similar to 

® Published by the author in the PM. Mag., vol, 19, 1936: pp. 389-402, 
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Exercise 1 except that now the y coordinates have unequal pre* 
cisions. Here we take 

F = y — a — hx 

as in the preceding exercise, the derivatives being also the aH.Tna 
But since x is free of error, Wx is infinite, and it follows that W 
(defined as 1/L) is none other than Wy, All we have to do is 
replace W in the preceding exercise hy Wy, and the results will 
apply here. The headings of Table 1 of Section 60 will he : 


}i, or Point No. 

Wy 

V Fi = -* 

For Table 2 they will be 

these: 

h, or Point No. 

VWi/ 

L = —x-\lwy Vu'v’Fo Sum 


The normal equations are written in the same symbols as those of 
the preceding exercise, but with Wy in place of W. Row III in the 
systematic solution of the normal equations (p. 168) gives the 
minimized value of 2 Wyiyoba — Veak)^- In Rows 5 and 6 are 
found the portions of the weighted squares removed by a and b, as 
in Exercise Ic (p. 175). 

Note that, as in the preceding exercise, it is permissible to take 
do and 6o as zeros, if the number of decimals in the normal equa- 
tions is increased accordingly. In this event, 

“ Voba 


and the normal equations may be written 


Row a 

b 

= 1 

Cl 

Ca 

Sum 

I N 

[ux] 

[wj?] 

1 

0 

■ ■ ■ 

2 

[uxx] 

[wxy] 

0 

1 

. . . 

3 


[wyy] 

0 

0 

... 


giving a and b directly. Row III in the systematic solution will 
give the minimized value of 2 ‘'OyiVoha — VeakY in the “ 1 ’’ 

' column, and Rows 5 and 6 will show the sum of squares removed 
successively by o and b, as in Exercise le. Since Wx is infinite, the 
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questioa of an approximate value of 6 for use in the calculation of 
W does not come up. 

Rmcark. For the conditions stated (x free of error) the 
Bum of the weighted y residuals is zero, i.e., 

» 0 

See Eemark 4 of Exercise 4, page 182. 

Exercise 6. (a) Given the tine 

y = a + hx 

to be fitted to n points, both x and y coordinates subject to error 
but in such a way that Wx/wy is constant and not infinite nor zero, 
the line passes through the center of gravity x = [in»x]/[iCj,], 
y = Kjf]/Wi with slope 

j, _ — [mi^] + + 4c[wuv]^ 


This is equivalent to a result obtained by Kmnmell in 1876, Karl 
Pearson in 1901, and Gini in 1921. Here u and v are the x and y 
coordinates of a point, measured from the center of gravity x, y\ i.e., 
iti = a, — X, and vt = yi — y. c is written for and w in 

place ,of Wt, for convenience. 

(6) If the plus sign be changed to minus in front of the radical, 
the result is the slope of the worst fitting line, that which maximizes 
the value of 2 (wxVz^ + WyVy^). 

(c) Prove that under these conditions of weighting, the best and 
worst fitting lines are perpendicular to each other. 

Exercise 7. (a) Given 

y = a + bx 

to be fitted to n points when y fe free of error and all x coordinates 
are of equal weight (unity), we may write 

x -v + qy 

and find the following normal equations for p and q. These are 
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Bow 

V 

2 

= 1 

Cl 

Ca 

Sum 

I 

n 

Ij/l 

M 

1 

0 


2 


[2/2/1 

[p>] 

0 

1 


3 



[xx] 

0 

0 



like Set 2 of Exercise 1 -with x and y interchanged. 

Row III in the solution of the normal equations gives £ res^ 
where now the deviations are measured parallel to the x axis. 

(b) The reciprocal matrix found in the Ci and Cs columns 
of Rows 7 and 8 of the solution will be 

1 1 1 JL ^ 

nsy^ 

1 

nsy^ 

where has the same significance as in Exercise 3. 

(c) (The S. E. of Xcau)^ = — |l + — — 

(d) The normal equations of Exercise 7a give £ = 0. (See 

the remarks in Exercises 1/, 4, and 6.) 

Exerdse 8. (a) Prove that with y free of error, and all x coordi- 
nates of equal precision, the normal equations for a and b (or fpr 
A and B) in Exercise 4 will give the same line as the normal equa- 
tions in Exercise 7a (i.e., will ^ve p = —ajb and 2 = 1/h)) except 
for the effect of the neglect of the squares of the x residuals. The 
solution of Exercise 7o is the more accurate in not throwing away 
any ^igliRv powers of residuals. This may occasionally be impor- 
tant. (See also Exercises 18 and 23.) 

(6) Show that if a; has the same weight (i.e., the same precision) 
over all n points, and y likewise, x and y both subject to error, the 
line that one gets by the exact solution given in Exercise 6a lies 
between the two false lines that one gets by i. throwing the 
adjustment all on to y, using the equations of Exercise 1; and ii. 
throwing the adjustment all on to *, using the equations of Exer- 


A-i = 


n 


nsy‘ 

V 


ns^ 


a 
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ciee 7a; but that these two false lines differ only in the effect of the 
squares of the x residuals and of the y residuals, respectively. 
{Hint: Both terms of 1/W in. Exercise 4 are constant over all 
points when w-c and Wy are constant; hence, so far as the values of a 
and 6 or p and q are concerned, 'W can be put equal to unity at every 
point in all three solutions — in the correct solution, and in the two 
false solutions. The normal equations of Exercise 4 will then give 
identical results for all three. But the normal equations of Exer- 
cise 4 can be in error at most by the neglect of higher powers of the 
residuals, hence the false solutions i. and ii. can differ from the 
true solution only through the neglect of such terms. This means 
that when x has the same weight over all points, and y likewise, 
the false solutions will hardly be distinguishable from the true 
solution if the residuals are all fairly sinall.^) 

Remark i. If is constant from one point to another, it is 
advisable for convenience of computation to choose the system 
of weighting so that F = I at aU points, This is only saying 
that the arbitrary factor o-’in Eq. 13 of Section 11 is to be chosen 
so that F = 1. Then S in the extreme left entry of Eow III 
in the solution of the normal equations comes out in the same 
system, and (r*(ca:{) = S/{n — 2) is the external estimate of tr* 
in the same units as were arbitrarily chosen for it. 

(c) All three lines of part (b) pass through tho center of gravity 

§ (called also the centroid). 

Remark %, Statements similar to those of part (b) will hold 
for any curve when the combination of the form of the function 
and the weighting of the coordinates causes both terms of 1/F 
to be constant over all n points. Example 3 of the next chapter 
is an illustration in three dimensions (three terms in 1/F). 

Exercise 9. For the line y = a + bx fitted to n points, the 
following expressions hold (all due to Karl Pearson, Phil. Mag., 
vol. 2, 1901: pp. 559-572). 

(fl) 2) res* = n(l — r®)s„® 

» free of error, the y coordinates all of equal weight (unity); 
the deviations measured in the vertical. (This result was given 
in Exercise 3b.) 

* This fact was noted by the author without proof in the Proe. Physical Sac. 
(London), vol. 47, 1936: p. 107. 



187 


[Ch. X] EXERCISES ON PITTING FUNCTIONS 

(b) £ res® = n(l - r®)8j;® 

y free of error, the x coordinates all of equal weight (unity) ■ the 
deviations measured in the horiaontal. ’ 

(c) £ res® = ^«(s,® + s„® - - Syy + 4r®s*®s„®l 

The X and y coordinates of equal weight (unity), the deviations 
measured perpendicular to the fitted line. 

In these formulas, s*®, s„®, and r have the meaning ascribed to them 
in Exercise 3, page 177. 


66. The parabola 

Exercise 10. Given 

y = o + 6a: + c*® 

to be fitted to n points, x and y having weights w* and Wy at any 
point. Here we take 

F = y ~ (a + 6a: + a®) 

Fo = Voba ~ (oo + ioXoba + CoXei,®) 


The derivatives of F are 


L or 


F^=-{h + 2cx), Fy = l 

Fa— 1, Fb~ X, Fo — —3^ 

1 (6 + 2cx)® ^ 1 

W Wy 


The headings of Table 1 in Section 60 will be these: 


A, or Point No. P* = — (6 + 2cx) v)* Wy L V'l' Po 


It is understood that in calculating all quantities under these 
headings, x and y are to be replaced by their observed values, and 
a, b, c, by ao, 6o, co (cf. tho note at the beginning of Sec. 66, p. 173). 
It is not necessary to tabulate Fy and Pa because they remain con- 
stant from point to point. 
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The headings of Table 2 -will be as shown below. 

h,OT PaHL Pb/'^L Eo/Vl' PoHL „ 

Point N o. =-Vr =-^W-x = -VTF-s'‘ = VTF.jg'o 

The usual process of cumulating sums of squares and cross- 
products in Table 2 yields tho following normal equations. 

Row A B C * 1 C\ Ci Ci Sum 

1 m \Wx] WA -[WFo] 1 0 0 ... 

2 [Fs®] [Fa“] -[WxFb] 0 1 0 ... (Set 1, 

3 [WA —Ws^Fo] 0 0 1 ... Exercise 10) 

4 +[FyoP'ol 0 0 0 ... 

The solution, carried out by the usual routine procedure, gives 
4, P, and C, whence the adjusted values of o, h, and c are 

fl s= Oq — 4 

b = bo - B 

c = Co — (7 

The minimized value of S or £ -f WyVy^) will appear in 

Itow IV, column *' 1,” This will be simply £ WiVa,® if y is free of 
error, and £ WyVy^ if a: is free of error. Directly above, in Row 8, 
appears the sum of squares that is removed from by shift- 

ing tho y intercept from oq to o; in Row 9 appears the further 
decrease brought about by allowing the second parameter to shift 
from bo to b; and in Row 10, just above 8, appears the portion of 
the sum of squares that is finally removed by adjusting the parsr 
bolic term from CoX® to cx® (see Exercise 3 of Sec, 01). 

The reciprocal matrix will appear in the Ci, C 2 , columns 
of Rows 11, 12, and 13 (the “ back solution ”), containing the 
variance and product-variance ooefiBcients for a, b, and c. (See 
Exercise 12 for the matrix A“^ in a fecial case.) 

Note the similarity between Set 1 of Exercise 4 (p. 179), and 
Set 1 of Exercise 10. Note also that if x is free of error, Oq, bo, and 
Co may be taken as 0, so far as Fq is concerned, in which event Fq 
becomes simply 

Fa = yob. 
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The normal equations then pvc a, b, and c directly, and would 
appear as shown below. 


Row 0 

b 

c = 

1 

Cl 

Cs 

Ce Sum 


I \w\ 

[F*] 

[Wx^ 

[Wy] 

1 

0 

0 

(Set 2, 

2 

[Wx^] 

[Fs’l 

[Wxy\ 

0 

1 

0 

3 

[Fi^l 


0 

0 

1 

Exercise 10) 

4 



[Wm\ 

0 

0 

0 



More dedmals will bo required here than if good valiicw of floi 
bo, and to had been used in the calculation of Fo, and the previous 
normal equations (Set 1 of this exercise) had been used to find A, 


B,andC. Why? 

The reciprocal matrix is the same, in both sets, and the min- 
imized value of Z + WyV/) again comes in the ex- 

treme left entry of Row IV ; but, as in llomark 5 of Exercise 4, 
the entries directly above it in Row.s 10 and 1) do not show tho 
inoraraents in the sum of the weighted .sciunres that would re.sult 
from setting c = 0 and b = c = 0, respectively, unless h and c 
are very email, or x free of error. 

Note tho similarity between Set 2 of Exornisc 4, and Set 2 of 
Exercise 10. Tlio remarks at the end of Exerci.se 4 apply hero 
with obvious modiri(iati(in.s. Eor example, approximate values 
of 6 and o must bo u.sod in the calculation of W at each point 
where !t is subject to error. 

Exercise 11. Given® 

y = a hz 


to be fitted to n points, x free of error, all y coordinates of equal 
weight (unity). If oq, bo, and co all be taken as 0, tho normal 
equations giving o, b, and c directly arc 


Bow a 

1 

c = 

1 

Cl 

Ci 

Ce 

Sum 

I n 


El 

HnH 

1 



< . « 

2 


HI 



1 

0 

. * . 

3 
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HSMH 

0 

0 

1 

. * . 

4 


■H 



0 

0 

... 


“ See the reduced type at tho boppuning of tliis chapter for roferonceB to 
special methods involving orthogonal polynomials, a[)plying to problems 
wherein x is equally spaced. 
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These arise immediately from the last set of normal equations of the 
preceding exercise by noting that under the conditions Wx = to 
and = 1 throughout, W = 1 throughout. Row IV in the solu- 
tion of the normal equations will contain the minimized value of 
E (yobs - t/caJo)® in the " 1 ” column. The sum of squares suc- 
cessively removed by the constant, linear, and parabolic terms will 
appear in the “ 1 ” column of Rows 8, 9, and 10 (see Exercise 3 of 
Sec. 61; also Exercises 1, 4, 5, and 10 of this chapter). 

Note the similarity between these normal equations and those 
of Exercise 1, Set 2 (p. 175). 

Exercise IS. (a) In the preceding exercise, let the origin of x be 
taken at the mean value of x, and let 71/12 be written for [i®] and 
n /14 for [a:*]. Then the normal equations are 
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The (S. E. of yea.icf at the center of gravity is equal to 
(rVw(l “ /*2V/'4)j which exceeds the value <^/n found in Exerase 2 
for the line. 

(6) Show that the determinant A of the coefficients is equal to 
nV 2 (M 4 — 1 ^ 2 )) hence that near indeterminacy (small A) will 
result not only in instability but also in high standard errors for a 
b, and c, and rapid fanning out of the standard error of ygaie (see 
Exercise 2c, p. 176; also Exercise 2o of Sec. 61, p. 162). 

67. The exponential and its logarithmic form 
Exercise IS. Given the equation 

y = ae^ 

to be fitted to n points, x free of error, all y coordinates of equal 
precision (unit weight). Here we take 

F = y — ae*’* 

Good approximate values of a and b can usually be found by 
plotting log y against x. Assuming that they can be obtained, we 
write 

Fa = Y — ooe^“* {Y denotes an observed y as in Fig. 17, p. 133.) 
The derivatives of F are 

Fy = l, Fa^ Fi= -xy 

Cl 

IF = 1 at all points; hence Tables 1 and 2 of Section 60 coalesce. 
They will be made up as follows. For convenience in writing, the 
subscript 0 will be withheld from the a and b. Xi, Yi, etc., are 
the observed x and y coordinates of the n points. 

Tablus 1 Aim 2 

h Fa Fh Fo I Sum 

1 -Yx/a 

2 -Yt/a 


etc. 
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The normal equations are formed from this table by summing 
squares and cross-products: 


Row 

A 

B 

1 

Cl 

C 2 Sum 

I 

iyVo*i 

[XY^a] 

-IFPo/o] 

1 

0 

2 



-[xrPol 

0 

1 

3 



[^'of’fll 

0 

0 


The solution of the normal equations by tho usual routine 
described in Section 61 gives A and B, also the reciprocal matrix, 
and the minimized value of £ res®. The adjusted values of a and 
h are then 

a = ao — A 
b = h-B 

The extreme left entry in Row III of the solution of the normal 
equations pves S or the residuals all being measured 

entirely in the vertical (i.e., parallel to Oy). Directly above 8, 
in Row 5, appears the sum of squares removed by tho shift from 
oq to a, and in Row 0 the further decrease brought about by adjust- 
ing the exponent from boX to ix. 

Exercise 14- If in the preceding exercise, the x coordinates are 
free of error but the y coordinates have unequal precision, desig- 
nated by weight w (varying from point to point), W is no longer 
unity, but is equal to w, which may vary from point to point. 
Table 2 of Section 60 then runs as follows: 


h 

FaNL 

FtHL 

■\lwFo 

Sum 

1 

-^IwiYi/a 

-XiFiVwi 

VwifFi — 


2 

etc. 

— VuigFs/a 

— XiFiV W2 

V WiCFi — 



The approximate values csq and 6o are inserted for a and b. 
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The normal equations, formed from Table 2 in the usual manner, 
^an be ^bolized as follows: 

Row ^ B = 1 Cl Ci Sum 

[ts7V«*l [loXF^/al -[wFFo/a] 1 0 ~ 

2 NX^F^l -bXFFol 0 1 I ... 

3 [wFoFol 0 0 ... 


i In the solution of the normal equations by the routine of page 
168, the minimized value of T. w{y - yeaicY comes in the 
extreme left entiy of Row III. Just above, in Rows 5 and 6, will 
appear the portions of the weighted sums of squares removed 
successively by adjusting a and then b (see Exercise 3 of Sec. 61; 
also Exercises 1, 4, 5, 10, 11, 12, and 13 of this chapter). 

The adjusted values of a and b are, as usual, 

a = Co — A 

b = bo -B 

A and 15 being found by solving the normal equations. Naturally 
these normal equations become tho same as those of Exercise 13 
if w = 1 throughout. 

Exerdae 15, (a) The formula to bo fitted in the two previous 
AvftrniaRH can be taken in the logarithmic form 

log y = log a + bx log e 

Suppose now that y' be written for log y, a' for log a, b' for 
6 logs; then 

= o' + b'x 

We now take 

/ = y' - (o' + b'x) 

/o = 7' - (oo' 4- bo’X) (7' = log 7 o 6.) 
whei'ein Oo means log oqi and bo moans bo log e. The derivatives 
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of / are as follows:® 


/ _ 7/ / _ 1 f _ 0-434 

/» 0 , Jy‘ — 1 , J]i — j I j 

dy dy y 

fa' = ~ 1 » fb> = ~® 


L I fxfx , fyfv 

or — 1 

W ^o« 


h'^ 0.434^ 

y'^v>y 


Suppose that « is free of eiTor, but that the weight of an observed 
y coordinate is «j, which may vary from point to point. (If all y 
coordinates have equal weight, it is easy to put w equal to 1 in 
what follows.) With Wa = Mj the first term in L drops out, 
leaving 

, 1 0.4342 

L or — = — 2 — 

W y^w 

It will be noticed from the result of Exercise 8e, page 45, that if 
to is the weight of y, then y^w/QAZ4? is the weight of y' or log j/. 
Suppose that on this account we set 

S 

Then v>' is the weight of y', and 

W ~w' 


If 10 (the weight of y) is constant throughout, then is not, and 
vice versa. (Cf , the remark appended to Exercise 18. ) 

Table 2 of Section 60 will have headings as follows: 


h, or 

Point No. 

ia‘/<L 

= 

h'NL 

w'-fo 

Sum 

The normal equations will be 

Row A* 

B' 

1 

Cl 

Ci 

Sum 

I Kl 

[aau'l -WM 

1 

0 

■ 1 • 

2 

[sV] -[sw'/c] 

0 

1 

■ • s 

3 

[w'/ofol 

0 

0 

... 


®It is convenient to remember that log e = 1/In 10 = 0.434 •• ■ = 
1/2.30 ■ ■ ■ . The symbol log means base 10, and the symbol In means base s 
(logarithms naturel). 
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We perceive that these are similar to the normal equations of 
Exercise 6, but with to' in place of w, and f in place of F. More 
precisely, the comparison is this: 

In Exercise 5, y = a + bx, x free of error, y of weight w. 

Here, y' = a' + b'x,‘‘ » “ “ y' “ “ 

This means that we may fit the equation y = oe*® by writing it 
in the logarithmic form 

log y = log a + bx log e 

and treating it as a linear equation in log y and x, at the same time 
giving In 2 / a weight just y^ times the weight of y, or log y a weight 
(2.30y)^ times the weight of y. 

Remark. It is customary among computers to fit the expo- 
nential equation y = oa*® by taking logarithms and treating 
it as linear in log y and x, but it is not so usual for them to 
change the weighting to correspond to the logarithms. The 
neglect of the factor (2.30j/)^ not only distorts tiie results for a 
and b, but also invalidates the reciprocal matrix and all calcu- 
lations made with it on the standard error of a function of the 
parameters; moreover, under such circumstances, the extreme 
left entry of Row III no longer contains 8. See also the re- 
duced type at the conclusion of Exercise 18, page 201. 

(b) The extreme left entry of Row III in the solution of the nor- 
mal equations contains 2 wi^obt — yoaie)^- The extreme left 
numbers appearing in Rows 5 and 6 are the weighted sums of 
squares removed successively by adjusting a and then b (see 
Exercise 3 of Sec. 61, and Exercises 1, 4, 5, 10-14 of this chapter). 
These statements would not be true if one were to neglect the 
factor (2.30y)® for the weight of log y. 

Note that in the normal equations of part (a) it is permissible to 
use Oq' = 0 and = 0, in which event 

/o = Y' or log Voba 


whereupon the normal equations will be as written below. 


Row a' b' 

1 

Cl 

Ci 

Sum 





■■ 












w' = 2.30®y®w as on the preceding page. 
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These normal equations give a' anil b' directly. As in Ex 9 ^ 
cisea 1 and 5, no qiiiMtion of tm approximtito value of b' enters for 
the calculation of the w', since w, is inlinito (a: free of error), but 
more decimals arc required than \vlu*.u good approximate values of 
a' and b' are used. (See Exercko Id.) 

In the solution of these normal cqiuitioim by the routine exhibited 
in Section 60, the mininiiiiMl l^vi(y„b, — f/cuic)* appears in the 
extreme loft entry of How III, a»s usual. Directly above it in 
Row 6 comes the reduction brought about by changing a from 1 to 
its final value, and in Row G appeal’s the further reduction aocom- 
pliahedby turning the logoritlunic line from the horizontal through 
the angle arc tan b'. 

Exercise 16. In fitting the equation 

y = Ofi’’* 

with X and y both subject to error, we may take F as in Exercise 13, 
whereupon 

Fo ^ V ~ (Xaud Eobiiorvcd) 

Here we have use for the additional derivative Fx = —by, whence 

r n. 1-^Vl 

w* 

If a: is free of error, tho first term of 1/W drops out and leaves 
W = tUv, the situation assumed for Exercise 14 ; if y is free of error, 
the second term drops out and leaves If = Wx/b'^y^. 

Since we are here taking the case whore x and y may both be in 
error, we set up Table 1 of Section GO with headings as follows: 

Point Lor ij Lor „ , „ n 

No. i/F l/VTT ° 

From this is formed Table 2 with headings exactly like those of 
Exercise 14 but with w replaced by If. Likewise, tho normal equa- 
tions will be symbolized as in Exercise 14, w replaced by If. In 
fact, once Table 2 is set up, from then on it is immaterial to the 
computer whether one or both coordinates aro subject to error — 
a statement that holds good in any problem of curve fitting. 
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The solution of the normal equations -will give A and B. The 
minimized sum of weighted squai'es (S) will appear in the extreme 
left entry of Row III, the portions removed by the successive ad- 
justments of fl and h falling in Rows 5 and 6 directly above B, 
(Cf. Exercise 3 of Sec. 61 and Exercises 1, 4, 5, 10-15 of this chap- 
ter.) Here, /S = £ both s and j/ residuals 

being present. 

Exercise 17. To use the logarithmic form of the exponential 
(see the preceding exercise) we write 

log 2 / = log a + h'x {b' = b log e 
= 0.434b) 


or 

y' = a! h'x 

for fitting the exponential y = ae'‘^ when » and y are both subject to 
error, one would define / as in Exercise 15; whereupon 

' fo = Y' — {oq' + bo'X) 

The derivatives of / are as in Exercise 15. L or 1/W will now 
have two terms, both coordinates being subject to error; in fact 

r 

W Wx Wyl 


Wyl being the weight of log y. The normal equations will be 
symbolized exactly like those of Exercise 15o, but with W in place 
of w'. The extreme left entry in Row III will be the minimiz ed 
sum of squares S, with the remarks at the end of Exercise 16 
applying here as well. 

The analogy with Exercise 4 is perfect throughout, as shown by 
the following summary: 


Exercise ^ 
y = a-\-bx 

i + l 

W w, 


Exercise 17 
y’ tz a' b'x 

1 = -V — 

W Wx W,ji 


rr M/* wj/ M 

All the remarks and notes of Exercise 4 apply here if y is replaced 
by y' or log y, a by a', b by b', and Wy by Wyi. 
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It is possible that in some problems, Wyi might be constant from 
one point to another (in which case Wy is not constant) ; then if Wx 
is also constant, we have a situation toward which Remark 2 at the 
end of Exercise 8 (p. 186) is directed. 

Exercise 18. (o) Take 

/ = log y — (log a -h h'x) (as in Exercise 15) 
F^y-ae^^ ( " " “ 13) 

and suppose that a, y, a, and b take on small increments denoted by 
Sx, etc. Prove that 

dF = ySf log e; 

hence at any point, Fq = yfo log e to within higher powers of fo or 
Fe. 

(b) Thence prove that the normal equations in Exercise 14 for 
fitting y - oe’’® will give the same curve, i.c., the same results for 
0 and b and for as the normal equations in 

Exercise 15o for the equivalent logarithmic form, except for dis- 
crepancies involving the squares and higher powers of residuab, 
the logarithmic form being slightly more accurate.^ (Hint: 
Note that if A is small. A' = 0.434/o. Also, so far as a and b 
are concerned, the top normal equation in Exercise 14 may be 
multiplied through by a.) 

The same comparison holds between Exercises 16 and 17. But 
with b 7^ 0, and x and y both subject to error, there is not so much 
advantage in the logarithmic form. 

Remark. It may be worth while to pause for a comment on the 
factor (2.302/)® which is requhed for the proper weighting of log y. 
Take the one-parameter curve® 

y = 10^ 

^ Mr. E. A. Norton pointed this out in one of the author’s classes. 

• This illustration was developed in some correspondence with Professor 
W. L. Gaines of the University of Illinois, extending between 1932 and 1938| 
also in conversations with Mr. G. R. Gause, lately of the Aberdeen Proving 
Ground, now with the War Department in Washington. 
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and to make the problem simple, let it be fitted to just two points 
in the xy x being free of error and both y observations of 
equal weight (unity for convenience). 

X y log y 

1 10 1 

2 95 1.9777 


To arrive at the least squares solution, we may write 
F = \ogy — 

Fo = log ydb> - 60® 


Fb = 

F^ = —bo (not needed because the weight of x is infinite) 

^ 

“ 2.30y 


«) = 1 

T _ ^ 

w (2.30J/7 


There is only one normal equation, namely, 

rFbFol Z w'x(log yob. - xh) 


wherein = (2.30 y)^. 

Note: The same equation for B can be derived by saying 
that we seek to minimize 

s = z w'Cios yoi. - log 

Replace log by bx, or, rather, its equivalent (60 - B)x, 

S = i:ta'(logy*.-[bo-5]x)* 

B is the (unknown) quantity which, when subtracted from 
bo, gives the final b. Now differentiate S with respect to B, 
and set this derivative equal to zero. The result is 

2 ia'a;(log yo6. — [f'o — B]x) = 0 
Solve for B, and the result will agree precisely with that shown 
above. 
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To continue, -we may take bo = 1, a value easily found by inspec- 
tion. We also replace w' by the correct weighting function 
and get 

_ 1 X 1Q’’(1 - 1) + 2 X 05^(1.97772 - 2 X 1) 

® “ 1^ X 10“ -h 2“ X 95“ 


402.15 

36,200 


= 0.01111 


whence 

b = bo - 5 = 1 - 0.01111 = 0.98889 

TMb is the least squares value of b. 

Now if in performing the solution we liad inadvertently token 
w' = 1, forgetting that the weight of log y is not the same as the 
weight of y, L would have appeared to be constant (unity), 
instead of proportional to j/*, and the result would have been 

„ E a Oog Vob, - xho) 



_ (1 - 1) + 2(1.97772 - 2) 

1“ -I- 2“ 


0.04456 


= 0.008912 


and b would have been 

• b = bo-5 = l- 0.00891 = 0.99109 

The comparison of the sum of squares for the two different 
solutions is shown below. 



Correct weighting 
b == 0.98839 


Folse weighting 
b 0.99109 



1 10 0.98889 9.7474 0.263 

2 96 1.97778 96.012 -0,012 


Sum of squares, S « 0.004 


log Ucatc — 
0.09109X 


0.99109 9,797 
1.98218 05.000 


Sura of squares, .S =» 0.974 
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Thus, by weighting log y in proportion to j/oiu® we obtain a sum 
of squares that is only a fifteenth that obtained by ignoring the 
change in weight. 

The only circumstance under which the factor j/* may be 
ignored is where the fitted bgarithmio line is nearly horizontal, 
for then the weighting factor is nearly constant from one 
point to another and can be omitted without serious error in 
the formation of the normal equations, the parameters a, 6, c, 
etc., being left practically unaltered. 

Even so, the lost entry in the " 1 ” column (Row IV, p. 168) 
is not S, but requires multiplication by an aver age value of 
(2.30y)®, which might be denoted by (2.30i/)*. Moreover, 
each element of the reciprocal matrix requires division by 
(2.30y)* before it is to be interpreted as a variance coefficient 
(of. the remark on p. 195). However, it is interesting to note 
that, owing to compensation, the uncorrected elements when 
used in Eq. 24 (p. 188), along with the external estimate of cr 
made from the uncorrected sum of squares, wiU give the correct 
value for the estimated standard error of a function. 

The factor j/* takes care of the change in scale that acoom- 
p fiTiteH the transfer to logarithms. The student may find it 
helpful to refer back to Hg. 9 on page 48. The y values nmy 
all have the same weight, but their logarithms do not. No 
matter what function is being fitted, the two terms 
and in L or 1/W (cf. Eq. 8, p. 134) can be relied upon to 

perform the some service as (2.30j/)* does for the logonthmio 

This «v°Tnp1p. is an illustration of the fact that if the pro- 
cedure of Section 60 is followed, it makes no difference how 
a formula ia written. One form will give the s^e curve as 
another, except for disturbances arising from the neglect of 
second and hi^er powers of the residuals, but these ore not 
usually of much consequence if the data are worth fittmg. 

Of course, in some lines of work, the weight of y is aiyroxi- 
mately inversely proportional to j/*, whence tte wa^t of log y 
is practically constant, independent of y. When this is so, the 
weighting factor j/* is to be omitted. 

Exercise 19. (Yntema’s refinement.)® In fitting the curve 

y = a^ 

“This device has been taught by Profeswr Theodore Y^tema ihe UM- 
versify of Chicago for years. It was firet called to my 
Smith of the University of Chicago (more recently of the Bureau of Labor 

Statistics in Washington). 
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with X free of error, we seek to minimize 
S = '^w(p- VcY 

where w is the weight of the observed y coordinate at a particular 
point, and Vo is the calculated ordinate, ae'’®. The normal equa- 
tions will be obtained by equating to zero the derivatives of S 
with respect to a and b, by which process we find that 

£«(!(-».)“ -0 

- 

Lw(y = 0 

d 

Now we may use the logaritlimic form by rewriting these 
equations as 

LBivW -y/)-^yJ -0 
i:e(,y)iy'-yc) |/2/c' = o 

wherein y' = log y, yj = log Ve, a' = log o, b'= b log e, and 
d(y) is such a function of y that the two forms of the equations are 
the same. Evidently it must be that 


dVo 

^ da y - ye 

Hy) = ^T~!~ ! 

dye y Vc 


dye V - Ve 


' '"dyev'-Ve 


da' 


= 2.30“wj/c*2/* {y hero denotes ijoba . ) 


The last equality is not exact, but is very close, as Professor Yntema 
discovered, and as the student may wish to demonstrate for himself. 

The normal equations are exactly like those in Exercise 15, d{y) 
now replacing w'. It will be observed that the Yntema refinement 
has merely replaced y® in w' by j/o®!/* in 6{y). There will be 
scarcely any distinction if the rraiduals are all very small, in which 
event ye and y (the calculated and observed y values) will be very 
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nearly equal, and will be very nearly the same as When 
the curve does not fit well, it may be important to take account of 
the Yntema refinement. 

68. The exponential with a linear component 
Exmiie ISO. Given the equation 

y = ae^ + cx + d 
to be fitted to n points. Write 

F s y ~ - cx ~ d 

The derivatives are 

= -e’’*, Fb = -xae\ = -1 

Fx = —abe'‘^ — c, F^ = l 

i , _1. 

W Wg Wy 

(The first term of 1/W is missing if a; is free of error, the second if y 
is free of error.) 

The formation of Tables 1 and 2 of Section 60, and the formation 
of the normal equations and their solution, proceed in much the 
same fashion as heretofore. The only novelty is that here there 
ore four parameters, and hence four unknown parameter-residuals, 
A, Bf C, and D. The numerical values of Fo and the derivatives 
Fa, Fb, Fx, etc., for use in Tables 1 and 2, are calculated with the 
approximate values oo, 6o, co, and do, arrived at somehow (see the 
reduced type at the end of Sec. 55). 

The extreme left entry in Row T of the solution of the normal 
equations will give the minimized -f- WyVy^). The 

entries just above it in Rows 12, 13, 14, and 15 will show the reduc- 
tions in the weighted sum of squares arising from the successive 
adjustments of a, b, c, and d. 

If only the y coordinates are subject to error, the extreme 
left entry in Row V will give)C ® the deviations being 
measured in the vertical (i.e., parallel to the y axis). Moreover, 
if all j/ coordinates have the same weight (unity), then TT = 1 
throughout, and Tables 1 and 2 of Section 60 coalesce. 
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With a formula of this Idnd, there is no possibility of making 
it linear by such a device as taking logarithms, for which reason, 
this problem and others like it have been called insoluble. For- 
tunately, the solution is entirely straightforward. 

69. The generalized hyperbola and its logarithmic form 

Exercise 81. Given the equation 

y =ax'‘ 

to be fitted to n points. Here we write 

F s y — crJ’ 

whence 

^"0 = Vobt — o^Xohi (fl and 5 being replaced by flo and bo) 

The derivatives of F are 

,, p — p _ 1 

— IT ’ ^ 

F„-~, Fi= -y\nx 
a 

r 1 , 1 

W a^Wj Wy 

The headings for Table 1 of Section 60 in the general case 
would be 


B 


L or l/W 

Vtorl/VW Fa = —v/<^Fb “ — 1 / las 


It is easy to make the necessary modifications for special cases. 
Thus, if a: is free of error, then W = Wy and the Fx and Wx columns 
are superfluous; if further, all y coordinates have equal weight 
(unity), then W = 1 for all poinia, and Tables 1 and 2 will coalesce. 
On the other hand, if y is free of error, then 1/W = and 

the Wy column is omitted. From Table 1 is formed Table 2 with 
these headings: 


V W'Pa or FaH It V W'Fi ox Fbf^L V W -Fo or Fq/ V It Sum 
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The usual sums of squares and cross-multiplications from 
Table 2 give the normal equations 


Row 

A 

B “ 

1 

Cl 

Ct 

Sum 

I 

[WFoFa] 

[WFaFb] 

-ITFFaFol 

1 

0 

. . . 

2 

[TTFiFsl 

-[WFM 

0 

1 

. . . 

3 



[irf*'oP’o] 

0 

0 

... 


Exercise 22. The equation y = ax* oi the preceding exercise 
may be turned into the logarithmic form 
log y = log a -I- 6 log x 


Qj y' = a' + ix' (as in Exercise 15.) 

Let / = j/' - (o' + i®') 

/o as usual 

The derivatives of / are 

0.4345 


, 0.434 

T‘ V 

U = - 1 . 


Then 


L or 


i, 0.434* + 


Wgl Wyl 


wherein 

_ 0 . 434 =^ ^ 1 

Wgi ” x^Ws wt. of x' or log X 


(See Exercise 8e on p. 46.) 


— similarly defined. 

Wyl 

will be these: 

The headings for Table 1 of Section 60 

h, or 

Point No. 

V)x 

L or 

W-l lOu ^ II 


(See the remarks under Table 1 of the preceding exercise. Ja' is not listed, 
being constEuit.) 
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The headings of Table 2 will be the usual ones, as in Table 2 of 
the preceding exercise with/ in place of F. 

The normal equations will be f^mbolized precisely like those of 
Set 1 in Exercise 4. In fact all the remarks and notes of Exer- 
cise 4 can be translated directly to the present problem. The 
reason is obvious; we have here a lino in the variables x' and y', 
with weights 10 */ and The two tcrniB of L or 1/W seen above 
take core of the change in the form of the function from exponential 
to logarithmic. In fact, wc could say that 


fx'fx' I fji'fy' 

W V)x> V)]/I 


Sxfx I fufv 
Wx W,, 


as in Exercise lib, page 46. 

Exercise SS. Prove that the normal equations of Exercise 21 
for fitting y = ax^ will give the same curve, i.e., the same results 
for a and b and hence for S, os the normal equations of Exercise 22 
for the equivalent logarithmic form, log y = log o -f b log x, ex- 
cept for discrepancies involving the squares and higher powers 
of residuals, the logarithmic form being slightly more accurate, 
especially if x is free of error. (Refer bock to Exercises 18 and 22.) 


70. The hyperbola with a linear component 
Exercise Given the equation 

y = ttx^ + c + dx 
to be fitted to n points. Write 

F^y-ax^-'C-dx 

Ffi&t any point is found, as usual, by giving x and y their observed 
values at that point, and o, b, c, d their approximate values oq, bo, 
Co, do (found somehow; Sec. 55). The derivatives of F are 

Fx = — — d, Fy = l, Fa=— Pb — — In ®, 

Fe = —1, Fd = —a 
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1_ ^ + df ^ 1 

W Wx Wy 

(W = ti;„ if a: is free of error) 

Tables 1 and 2 of Section 60 are made up, and the normal equa- 
tions formed and solved, by the usual routine. Row V in the " 1 " 
column will give the minimized value of S or £ + WyVy^), 

the successive reductions in the weighted sum of squares appearing 
in Rows 12, T3, 14, and 16, as usual (see Exercises 1, 4, 6, 10-16). 

The reader should refer back to the reduced type appended to 
Exercise 20, which applies here as well. 

Exercise SB. Given the equation 

u = ax by‘ + ds 

u, X, y, and a possibly all being observed. (This equation is used 
by Professor W. L. Gaines at the University of Illinois in his work 
on nutrition and lactation.) Take 

F = u — (fix + by” + dz) 

Ffj as usual 


The derivatives of F are 

Fu =1, -a, Fy = -dby'=-\ F, = -d 

Fa = -X, Fb = -y% Pc = -by" ^y> Pd * 

W Wu Ws '^V 


Here we have a problem in four dimensions) 1/T7 contams four 
terms. The first term is absent if « is free of error, the second if x 
is free of error, etc. 

The headings for Table 1 in Section 60 would be these: 


h, or Wu Wx by° Pv 

1/W 1/VT7 

Point No. 



Fa = 

Pb = 

F.= 

Fi= 

Fo 

—a: 


-by"!!!]/ 


■ 


Some of these headings will be omitted if any of the u, x, y, or a 
values are free of error throughout. 
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Table 2 will be formed by divisions in Table 1, and the headings 
would be as shown below: 


h, or 

Point No. W'Fa ■>IW‘F, ■ylW-Ft ■4W-F^ Sum 

The normal equations will be formed and solved in the usual 
routine manner (p. 158). Row V will oontain the minimized value 
of S, being in this case 2) + '^iizYx + WyV^^ + w,y/) 

in the “ 1 ” column, the successive reductions in the weighted sum 
of squares appearing in Rows 12, 13, 14, and 15, as usual (see 
Exercises 1, 4, 5, 10-16, 24). 

If only the u coordinates are subject to error, »Sf = £ w^V^. 
Moreover, if all the u coordinates have the same weight (unity), 
then ly = = 1 throughout, and Tables 1 and 2 coalesce. 

The second paragraph in reduced type appended to Exercise 20 
applies here (p. 204). 


71. Miscellaneous 

EzerciseSd. (a) Given the equation 
u = ax + by + cz 

to be fitted to n observed poinis. Take 

jP = « — {ax by -\r Gz) 

and show that 

W Wu tOs Wy ' Wt 


Then the normal equations will be symbolized in the form shown. 


Row 

A 

B 

C = 

1 

Cl 

<7* 

Ct Sum 

I 

[PT's®! 

[Wxy] 

[Wxe] 

-[TTiPo] 

1 

0 

0 

2 


IWyy] 

[TTi/b] 

-IWol 

0 

1 


3 



[TFsal 

-[WzFa] 

0 

0 

1 

4 




[WFoFo] 

0 

0 

0 
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( 6 ) If it is desired to solve for a, b, c directly, the unknowns in 
the normal equations would be a, b, and c, and the “ 1 " column 
would be 

[Warit] 

\Wyu\ 

\Wsv\ 

[pr««] 

Extra decimals will be required for accuracy, as mentioned on 
page 175. 

(c) Prove that the minimized sum of the weighted squares is 
S = [WFoFa] - [WxFM - [WyFolB - [WzFo]C 

If the normal equations are set up to give o, b, and c directly, as in 
part ( 6 ), then 

S — [TFttu] — — [Wyu]h — [Pr«u]c 

(See Exercise 3a in Sec. 61.) 

Remark. If u alone is subject to error, and of uniform pre- 
cision (unit weight) throughout, the only change in the normal 
equations would be that IT would not appear, being unity 
tluoughout. The minimized sum of squares would be 

jS = [«m] — [®«]a — ]yu]b - [zu]c 

This equation is used a good deal in some kinds of statistical 
work; see, e.g., p. 100 of the 6 th edition of Fisher's StalM- 
ticdl Methods for Research Workers, on which the above equation 
appears as 

S(y - y)!i = S(j/2) - hiSipny) - btSixiu) - baSCxiy) 

this being the sum of squares after fitting 
Y = 61 ®! -|- bjZj -j- bsXs 
Example 3 in Chapter XI is an illustration. 

Exercise S7. In pharmacology and toxicology, experiments are 
made on a certain number n of organisms or animals to test the 
lethal action of a drug or dosage of X-rays, for various concentrsr 
tions, or various times of exposure. The proportion killed is 
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usually designated by the letter p, and the proportion survi-ving 
by the letter q. Under the assumption that the susceptibility of 
an individual to a poison is a normally distributed variate, the 
relation of p and q to the deviation y from the average suscepti- 
bility may be expressed in teims of the normal integral by the 
equation 


? = 



dt=l-p 


By using a table of the normal probability integral it is possible 
to express q in terms of y. To avoid the use of the negative normal 
deviates that arise when the observed survival q is more than half 
of the animals tested, Bliaa^° has introduced the term probit, and 
has provided tables for conversion. Probits are simply normal 
deviates to which the constant number 5 has been added. If Y 
denotes a probit, then Y = y + 5. The scale in probits runs 
practically from 1 to 9, the S in the middle corresponding to the 
center of the normal curve, where y = 0. 

It has been found that when the dosage is expressed in loga- 
rithms, and the observed proportion q surviving is transformed 
into probits, then the relation between the log-dosage and the 
probits surviving is approximated by a straight line. The fitting 
of this line, with proper weighting of the points, constitutes an 
important application of least squares. To fit the line by least 
squares, the weights of the probits must be obtained. Now q is 
a proportion, and the assumption is made that the n ani mHl s or 
organisms are drawn randomly from some universe, wherefore the 
theoretical variance of q is pq/n. Then by Eq. 8 on page 40, the 
variance of the probit Y can he written 



Show by differentiating the equation relating qtoy that 


„ 2 _ „ 2 _ 

Oy — cry 2 


“ C. I. Bliss, " The calculation of the dosage-mortality curve, ’’ The Anmk 
qf Applied Biology, vol. 22, 1936; pp. 134-167. 
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i3 tie waglit to be applied to tie piobit F in itting tie dosage- 
mortaity cmve. Note tkt tie piobit F ie a quantity called a 
penmie of a distiibution. If tie standard deviation of He 
ffl ni plftt imiveisoiav, then tie 



Since piobits an by deinition expressed in tenns of the standard 
deviation of the assumed normal curve, the quantity in this 
problem is qual to unity. 



CHAPTER XI 

FOUR EXAMPLES IN CURVE FITTING 


ExamfiiB 1. Fitfinq an Ibothbbm 


72. Fonnatlon and solution of the normal equations. For thia 
example, data taken by the Michels^ et al, on carbon dioxide will 
be used, The equation to be fitted is 


y = a + hx + cx^ + dx*‘ 


(y denotes py, pressure times 
volume; a: denotes density) 


The parameters are not independent but are subject to the condi- 
tion that 

y = 1 when 1=1 


This condition arises because of the definition of the unit of volume. 
Because of this condition, 


c=l-b-c-d 


K = 1 + (» - 1)6 + (x® - l)c + (i* - l)d 

Weights: All y coordinates have equal weight; i is free of error. 
Let 

F = p - (1 + (» - 1)6 + (x® “ l)c + {x^ - l)dj 
Derivatives: 

Fi= + 1, Fo = +1, Fd= -X* -f- 1 

Fy = 1, Fx is not needed since x is free of error. 

PT = 1 at every point. 

^ A. Midhds, C. Michels, and H, Woutcrs, Ptoc. Royal Soe, (London), 
vol. 163A, 1038: pp. 201-224. 
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The following approximate values are known from previous 
experience: 

6o = -0.006837046 
Co = 0.000011392 

do = 0.0® 1514 

Then 

Fa = Vob, - {1 - 0.00683706(* - 1) + 0.000011392 (»= - 1) 

+ 0.0® 15(s^ - 1)1 

TABLES 1 AND 2 

(FoHMBD mOM THE OBleiNAli DA.TA) 


Point No. 

-Pi 

-K 

-Fd 

-Fa 

Sum 

1 

1.77X10 

3.61 X10> 

1.24X10® 

0.66X10"® 

7.07 

2 

2.26 

6.49 

3.03 

0.79 

11.66 

3 

2.72 

7.93 

6.30 

2.07 

19.02 

4 

3.18 

10,74 

11.66 

3.00 

28.66 

5 

3.66 

14.16 

20.04 

6.22 

44.07 

6 

4.14 

17.99 

32.39 

10.06 

64.67 

7 

4,61 

22.20 

49.31 

14.84 

90.00 

Sum 

22.33 

82.01 

123.86 

37.61 

266. 81 V 


In this example, W = 1 throughout, with the result that Tables 1 
and 2 mentioned in Section 60 are identical. The minus signs in 
the headings avoid minus signs in the table. The powers of 10 
bring uniformity in the denominations of the columns. 

The original data were listed to more decimals than are in- 
dicated by the above table, and the normal equations shown 
here, it so happens, were formed from the original data, retaining 
aU decimals, then rounding them off to the number shown. 
Exact agreement con not be expected, therefore, with the accu- 
mulated squares and cross-products that one would form in the 
usual manner from the table above. The effect on the param- 
eters, arising from the use of the extra decimals, is negligible, 
and the conclusions are the same either way. ' 

The Sum oolumn provides a check, which should never be 
omitted; it is formed regardless of the powers of 10; in fact no 
attention is paid to the powers of 10 until the end, when the 
solution is decoded. After the normal equations are formed, 
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the sums in Rows I, 2, 3 are each raised by 100 to take account 
of the entries in the C columns. 

Why is it better to start off with 100 rather than 1 in the 
C columns, for the calculation of the recipi-ocal matrix? Per- 
haps 1000 would have been better than 100. _ 

Note the symmetry in the reciprocal matrix, which is found 
between the .vertical lines in Rows 11, 12, and 13. 

From the normal equations one may make up the following 
tabulation of results. 

+0.036X10"® h= -0.00683705-0.0*36 = -0.00683741 

(1 0.015 X 10"® c = 0.0^1392+0.0^16 = 0.0^11407 

D=+0.307X10"® d= 0.0*151-0.0*307= -0.0*156 

a==l_j,_c-d=1.00682600 

Est’d S.E.* of b = 236.75 X 10"*"V(ea:<) = 19.3 X 10"‘* 

« “ » c = 41.65 X l(r*"*ff*(ea!0 = 3.40 X 10"^* 

II It « d = 2.56 X 10"*"^®o-*(ea:0 = 0.209 X 10"*° 

« « “ 0 = ff* (6x0(236.75 X lOr^ + 41.66 X 10"® 

+ 2.65Xiri*- 2 X 98.03 XlO-® 

+ 2 X 22.98 X 10"° - 2 X 10.08 X 10"°) 

= 17.7 X i(r“ 

See Eqs. 21 and 22, p. 167; remember that o is a function of 6, o, 
and d. 

See also Exercise 1 ahead. 

Final results for the parameters: 


0 = 1.0068260 ± 0.0000042' 

h = -0.0068374 ± 0.0000044 
c= 0.0*1141 ±0.0°18 [ 

d = -0.0°156 ± 0.0°046 J 


Standard errors estimated 
from 4 degrees of freedom. 


These standard errors appear to be small compared mth the 
Hc^ve,, it n.«t U ^ that th^ 
from only 4 degrees of freedom. There is really not much that one 
irom only g nrediction of future data, purely on the 

caa say m the way of the preoiomou n qmffle 
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degrees of freedom, as is true hero. A consistent pattern of AmH.]] 
standard errors, in experiment after experiment, would be^n to 
assume scientific significance, and such is in fact the actual situa- 
tion with compressibility data, though the other experiments, and 
the calculations therefor can not be shown here. 

An important consideration was voiced at the outset in Chapter 
VIII, wherein it was stated that the real test of a calculated curve 
comes when it is used as a basis for action. The form of equation 
used here, and the method of fitting, have been tested severely in 
this way. For instance, by means of this equation, the Michels 
have calculated various physical properties of carbon dioxide, and 
they and others have carried out similar calculations for other 
gases, and always the results of these calculations have tied up 
closely with whatever dhnet experimental work exists on the 
index of refraction, Joule-Thomson coefficient, heat capacities, 
entropy, and other properties, most of which are difficult to meas- 
ure directly. Manufacturing processes designed on the basis of 
these calculated physical properties have turned out to be correct, 
thus bearing out the usefulness of tho parameters so calculated. 

This statement does not contain any argument tliat these par- 
ticular parameters are better than any other sot that could be 
obtained from the given obseiwations. It would take a long run 
of experience in the use of various alternative procedures for fitting, 
in order to decide just what method is better than another. Such 
comparisons probably do not emst. 

It is interesting to see what would be the sum of squares if the 
term d®* had been dropped. Prom Row III we find [oo.2] = 
39.16 X lO*’”; this multiplied by d* or (“-O.O^ISG)** gives 0.9 X 
10~*, which added to 0.33 X 10~® gives 1.23 X 10~® for the 
sum of the {yot — Voau)^ that would be obtained from fitting the 
curve y => a + hx-^ cs*. We then find 

10 -® 

O'® (eat) =s 1.23 X = 0.266 X 10"® (d®* omitted) 

0 

We already had 

(r*(ac0 = 0.082 X 10"® (d»® included) 

Since the sum of squares, and hence er(eE«)i is so much lower 
when the term d®® is included, it appears from the meagre 
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evidence afforded by the four degrees of freedom of this one 
experiment, that one is warranted in carrying this term. 

One may also use Eq. 17 on page 166 for computing the effect 
of dropping d. One thus finds 

O’- 

Exercise 1. Show that when corrected for powers of 10 the 
reciprocal matrix is 

236.75 X 10“* -98.03 X 10"® 22.98 X 10"® 

^-1 = -98.03 X 10"® 41.65 X 10"® -10.08 X 10"® 

22.98 X 10"® -10.08 X 10"® 2.56 X ir^® 

These are the figures that were used in writing down the standard 
errors of a, b, c, and d. Evaluated as a determinant, this gives 
A"^ = 4.6 X 10"®®. 

Exercise 8. The evaluation of the determinant of the coefficients 
is 

A = (77.53 X 10®) (53.30 X 10^)(39.16 X 10^°) = 0.162 X 10®® 

(See Exercise 1 of Sec. 61, p. 161.) This result is not exact; the 
discrepancy arises from instability, and could be overcome by 
carrying more decimals. 

Exercise S. (a) Prove that the standard error of the curve at 
a: = 1 is 0, and that at a: = 0 it is the same as the standard error 
of o. 

(6) Why is the standard error of the y mtercept practically equal 
to the standard error of b? Argue geometrically and analytically. 

73. A note on instability. As often happens in curve fitti^, 
these normal equations are imstable. One of the moat sensitive 
tests for instability is to compare the direct solution (already found 
in the " 1 ” column of Rows 11, 12, and 13) with that given by 
iiaiu g the reciprocal matrix as a multiplier; by such means we get 
the reciprocal solution (pp. 166-166) 

10B= {151.06 X236.75 - 654.02X98.03+ 1233.8 X22.98}l(r®"® 

= 0.0237X10"® 
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and in like manner (which the student should undertake as an 
exercise), 

10®(7 = -0.0508 • 10“^, 10®2) = 0.2500 • lO"^ 

These are in disagreement with the direct solution found in Rows 
11, 12, and 13, and thus instability is indicated. The direct solu- 
tion satisfies the normal equations to the last decimal, but, when 
there is instability, many other solutions not too far away could 
do the same thing.^ The reciprocal solution, however, does not 
satisfy the normal equations, the actual numbers being 111 against 
151.06, 483 against 054.02, 919 against 1233.79. As indicated in 
Exercise 2, these discrepancies could be overcome by carrying 
more decimals. 

The insidious thing about instability Is that its presence may go 
undetected. For instance, if here we had only the “ reciprocal 
solution,” and had not tried to check it by substitution, we might 
have accepted it. The use of the reciprocal matrix as a multiplier 
is in theory very fascinating, but as a practical mattetr in curve 
fitting we should not wax too enthusiastic about it. Fortunately 
it does work to good advantage in many problems, as seen for 
instance in Chapter V of R. A. Fisher's Statistical Methods for 
Research Wark&'s. In Section 36 also, the equations were 
stable and no difiiculties arose. 


Example 2, Anotheb Polynomial 

BOTH X AND y OBSERVATIONS SUBJECT TO BEHOR 

74. The observations and their weights. The polynomial 

y = a + hx + cx^ (1) 

is to be fitted to points in the xy plane, x and y both subject to 
error. The observations on the coordinates are shown in the 
accompanying table. 

*See footnotes 10 and 12 in Chapter IX (pp. 160 and 161) for references 
to Tuckerman's paper and a note by die author on this subject. 
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Table op observations (example 2) 

iV denotes the number of observations at a point, s their standard deviation, 
defined by 

s* = - »>® “ ^ r ree’^ (3) 


with a similar equation for the standard deviation of the f/ observations, 


Point 

No. 

h 

Pot the ss coordinates 

For the y cooidinates 

True 

value 

Obs’d 

If 


True 

value 

Obs'd 

N 


1 

-2 

-2.28 

5 

0.154 

0.11 

0.129 

10 

0.00447 

2 

-1 

-1.13 

6 

.162 

.16 

.131 

9 

177 

3 

0 

-0,44 

7 

.202 

.20 

.198 

8 

264 

4 

1 

1,44 

8 

.316 

.26 

.247 

7 

182 

6 

2 

1.90 

9 

.176 

.31 

.312 

6 

ns 

6 

3 

2.93 

10 

.124 

.38 

.380 

5 


7 

4 

3.81 


.307 

.45 

.441 

7 

294 

8 

5 

S,07 


.032 

.63 

.629 

12 

286 

9 

6 

6.11 

10 

.343 

.61 

.690 

4 


10 

7 

7.17 

9 

.016 

.70 

.728 

4 

082 

11 

8 

7.83 


.176 

,79 

.701 

7 

244 

12 

9 

9.32 


,164 

.89 

.922 

5 

442 


76. A note on the observed values. This is an artificial example, 
carried out under ideal conditionB, in order to combine special 
features of a number of practical examples that might have been 
chosen for illustration. The coordinates observed (the “ true ” 
points) were taken along the curve 

2/ = 0.2 + 0.05® + 0.003®® (3) 

The standard error of a single observation on an ® coordinate was 
assumed to be 0.6 cm., and the standard error of a single observa- 
tion on a 2 / coordinate was assumed to be 0.05 lb. Artificial 
observations were taken by using Tippett’s numbers, assuming 
that the observations are normally distributed, according to the 
table shown in the appendix. Considerable departure from the 
normal distribution would not affect the results appreciably. The 
procedure can be described as follows: 
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ISeo. 76] 

i. Read out N mimboi's systematically (i.e., read up, or 
down, or diagonally) from Part A of the appendix (p. 2S2). 

Each number is a deviation, in units of Ci or o-j, as the case 
may be, from the true value of whatever coordinate is being 
observed. If desired, Tippett’s Random Sampling Numbers 
(Traets for Computers, No. 15, Cambridge 1027) can be used, 
in conjunction with, the table in Part D of the appendix. N is 
the number of observations on a coordinate, as shown in the 
accompanying Table of Observations. 

ii. Talce the average of these N deviations, and multiply it by 
ff* if on a: coordinate is being measured, or by o-„ if a 2 / coordinate 
is being measured. (The numerical values of Cx and are to 
be discussed shortly. Assume for the moment that they have 
been settled upon.) 

iii. Add this deviation to the true coordinate to got the ob- 
served coordinate, and enter it in the table. 

iv. Compute the variance, or the square of the standard 
deviation, for the observations on each coordinate. These are 
shown as in the table. Tlie formula is in the heading. 

The question arises how to weight the various values of X and Y. 
For one thing, the weight of any coordinate will bo proportional 
to N, but that is not enough; the precisions of single observations 
are evidently not the same for the y coordinates as they are for 
the X coordinates, judging from the s^ columns. In order to check 
the X and y precisions for this particular sot of obsei’vations, as one 
might wish to do in practice, wo may plot Fig. 21 to show the 
Buflceasive values of / (JV — 1) for x, and of the samo thing for y, 

both plotted against x {y would do as well). 5®iV/(JV — 1) for x 
(or y) at any point is an estimate of tho square of the standard 
error of the single observations on x (or y) at that point. 

Although there is fluctuation of the estimates, there is not too 
much, and there is no trend.® Now the weighted average on 
the X plot is not far from 0.25, and the weighted average on the y 
plot is not far from 0.0026, so it seems reasonable to conclude 
that the prior values of precision (viz., o-* = 0.5 cm., ay = 0.051b.) 
should not be changed. In practice, standard errors are usually 

' In pToctice one must have enough ectimatcB to enable him to plot a 
Shewhait control chart, before making such statements. However, here we 
have a method (the use of Tippett's numbers) that in the pnat hos demon- 
strated randomness, and these etatoments can safely be made. 
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known pretty definitely from previous experience. Accordingly 
■we take 0.5 for the standard error of single observations on » 
and 0.05 for the standard error of single observations on y, over 
the entire range. By recalling that weights are inversely propor- 
tional to the variances (p. 21 ), we see that a single observation on 
a y coordinate has 100 times the weight of a single observation 
on an x coordinate (Eq. 16, p. 22). As a matter of convenience, 



Fig. 21. Estimating the prooisiona of the observations. The chart shows 
estimates of tho squares of the standard errors of single obseryations. A c^ait 
of this kind will disclose trends and abnormal variations in the precisions, 
though one should have more points than this at his disposal. 


then, we talte unity for the weight of a single observation on x, 
and 100 for the weight of a single observation on y. ^ This is 
equivalent to setting v = 0.5 for observations of umt weig t. 
The values of 10 ® are then the same as the numbers N referrmg 
to X, in the table of observations, and the values of % are 100 
times the numbers N referring to y. (If the precision of smgle 
observations on either x or y coordinates were variable over toe 
range of the points, obvious modifications could be made m toe 


weighting.) .. „ 

76. Formation and solution of the normal equations. We shaU 

cany out the steps called for in Section 60. ^ 

1st step: get approximate values for a, I, 0 . By paasmg e curve 
through three selected pointe, approximate values for oq, 60 , 
Co could be found (see the reduced type on the method of seleote 
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points, Sec. 55). In tliis particular case, however, we shall 
instead use the true values of the parameters as approximations. 
They are found in Eq. 3. Accordingly, we write 

Oo = 0.2 'j 

bo = 0.05 i- (4) 

Co = 0.003 J 


Of course, the final results will be the same, no matter what values 
we choose (within reason) for the approximations Oq, 6oi cq (of 
the reduced type on pp. 137 and 138). 

Sd siep: the dmvaMues. For the function being fitted (Eq.l) 
we write 

F = 2 / - (o + ba: 4- ta*) (6) 


and then find the following derivatives: 

F,= -(6o + 2cqZ), F„=1 
Fa= -X, 


whence (see Eq. 8, p. 134), 


L 



(bo_+2eo^_l_J_ 

Wx Wy 


Also we write 

Fo = y - (flo + hoX + coZ*) 


( 6 ) 

(7) 

( 8 ) 


Sd step: numerical values; Table 1. We are now ready to 
calculate the numerical values of 1/W, 1/ V W, Fa, Fb, Fc, and 
Fo, and to compute Table 1, which precedes the matrix, Table 2. 

step: preparation of the mairix] Table 8. Now divide the 
values of Fo, Fi, F^, and Fq in any row by the corresponding value 
of 1/V W, and form the sums at the right and at the bottom for 
checks. 

BOi step: ihe formation of the normal equations. The normal 
equations are formed by the usual accumulation of squares and 
cross-products from Table 2. The solution is carried out by the 
routine outlined on page 158, and used previously on pages 82 
and 83, and in the preceding example. 
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way through, it would ordixuuiiy not be listed in a column. 
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TABLE 2 


TBB POUHTH BMP! THB MATMX TOB TBB FOBMATION OF THE NORMAL 
EQUATIONS 

{This comes by divisions performed on Table 1.) 


Point 

No. 


-^|W-Fs 

-VF-E. 


Sum 

1 

28.13 

-0.413GX1O 

1.4623X10® 

+7.7104X10-* 

30.8801 

2 

26. G2 

-2.9908 

0.3380 

-4.3307 

19.6311 

3 

26.24 

-1.1100 

0.0480 

+4.9016 

29.0796 

4 

23.20 

3.3408 

0.4811 

-7.2430 

10.7780 

6 

21,90 

4.1610 

0.7000 

+1.3612 

28.2028 

6 

20.17 

6.9098 

1.7310 

+1.6032 

29.3748 

7 

21.38 

8.1458 

3.1035 

+1.4838 

34.1131 

8 

20.20 

10.2616 

6.2026 

-0.4080 

36.2461 

9 

17.54 

10.7160 

6.6481 

-4.8236 

29.0816 

10 

17.00 

12.1890 

8.7306 

+2.6060 

40.5244 

11 

18.09 

14.8692 

11.0426 

+2.9667 

48.4686 

12 

16.36 

14.3002 

13.3334 

-0.7040 

42.2860 

Sum 

265.02 

73.3692 

63.4228 

5.0630 

387.4660V 


Cleared of miauB signs and powera of 10, Rows 11, 12, and 13 
lead to the following values of the parameter-residuals A, B, C, 
and to the reciprocal matrix ^own below. 


A = -0.00241 
B = 0,004306 ■ 

C = -0.000577 




0.0’'2767 0.0®3 -0.0®57 

0.0®3 0.0*638 - 0.0®84 

-0.0®67 -0.0®84 0.0«14 


( 9 ) 


( 10 ) 


The adjusted parameters will be found by subtracting each 
residual from the corresponding approximate value, according to 
Eqs. 6 in Chapter IV, page 62. The numerical results follow. 




> 


o = oo - A = 0.2 -I- 0.00241 = 0.20241 

h = bo-B = 0.05 - 0.004306 = 0.045694 

0 = cq~ C = 0.003 + 0.000577 = 0.003677 


( 11 ) 
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The factor 10“* holds all the way down the column. 
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The reciprocal matrix contains the variance and product varian 
coefficients, whence we may write the standard errors of ih 
parameters a, b, c, and of any calculated y value or, m fact f 
any function of a, 6, c. The diagonal shows that the’ ’ 

(S.E. of af = 0.00027670-2 ’ 

(S.E. of hf = 0.0000038<r3 /in^ 

(S.E. of c)2 = 0.0000014ff2 ^ ’ 

/ 


With a ~ 0.5 (the standard oiTor of obscrvatiomi of unit wei(rht\ 
it follows that the ® " 


(13) 


(14) 


S.E. of a = 0.0083 
S.E. of 6 = 0.0040 
S.E. of c = 0.00059 

whence 

a = 0.2024 ± 0.0083 
h = 0.04569 ± 0.0040 
c = 0.00358 ± 0.00059 

77. The reciprocal solution. The reciprocal solution for the 
unknowns A, B, G is obtained os follow, s: 

-A = 147.18 X 0.0277 - 29.52 X 0,0003 

- 28.95 X 0.0572 = 0.0241 X IQ-^ 

-lOB = 147.18 X 0.0003 - 29.52 X 0.6382 

- 28.95 X 0.a381 = -0.4306 X 10“‘ 

-lOOC = - 147.18 X 0.0571 + 29.52 X 0.8381 

+ 28.95 X 1.4275 = 0.5766 X lO-^ (17) 

These values of A, B, and C substituted into the left-hand 
members of the normal equations give numbers that are to be 

compared with the right-hand mcmbei-s. The results are shown 
below. 


(15) 

(16) 


Row 

Value of the left- 

Value of the right- 


hand momber 

hand member 

I 

147.27 

147.18 

2 

-20,00 

-29.62 

3 

28.98 

28.96 
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This close agreement, however, required more figures in Tables 
1 and 2 were advised on pages 79 and 155. 

78. Adjusting the observations. The calculated point cor- 
responding to the observed coordinates X, Y can now be found. 
We note that there will be a X at every point, given by Eqs. 10, 
page 136. 

Xi = WiiFo^ - - F6*5 - Fo'C) 

The superscripts on F refer to the point numbers, as th^ did on 
page 133. We use A = -0.00241, B = 0.00431, C = -0.00058, 
with the values of Fo, Fa, Fb, and Fo already entered in Table 1, 
page 223. We shall adjust the observations only at Points 10, 11, 
and 12, for illustration. 

^ {0.01527 - 0.00241 + 7.17 X 0.00431 

- 51.41 X 0.00058} = 4.029 


Xio - 


Xu 


Xl2 — 


0.003461 

1 

0.002772 

1 

0.004244 


(18) 


{0.01557 - 0.00241 + 7.83 X 0.00431 

- 61.31 X 0.00058} = 4.094 (19) 

{ -0.00459 - 0.00241 + 9,32 X 0.00431 

- 86.86 X 0.00058} = -4.055 (20) 

whence, by applying Eqs. 12, page 138, the residu^ can be com- 
puted at once. The required values of F^ are m Table i. 

At point 10, 

y =1 xiof** = ^ X 4.029 X -0.09302 = -0.0416 
1 4.029 _ 

At point 11, 

y = 1 X 4.094 X -0.09698 = -0.0567 


(21) 




7 

4.094 

700 


= 0.0059 


( 22 ) 
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At point 12, 




-4.055 X - = 0.0859 


-4.055 

BOO 


-0.00811 


(23) 


V 



Fio. 22. An illustration of adjusted observations in curve fitting. The 
calculated curve and the 96 percent error band arc shown in the neighborhood 
of points 10, 11, and 12. The error bond is laid off above and below the col' 
culated curve. The calculated or adjusted points lie on the calculated curve, 
exceptfor'‘error3of closure." Compare with Figs. 16 and 17,pagesl32andl33. 
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These residuals measured off from the observed points in the 
proper direction (see Fig. 22) give the cdcuhted pmnts, which 
lie on the calculated curve. Actually the points so calculated here 
do not fall exactly on the curve. Such discrepancies are trifling, 
being of second order from the neglect of second and higher powers 
of the residuals. One may simply manipulate the end decimal 
of 7* or Vy or both, in order to place the calculated point 

! _ V w* I (^1®- P- 138) 


exactly on the curve. A precisely similar situation arises in 
problems of surveying wheran, for exact satisfaction of the 
geometrical conditions after adjustment, one often needs to manip- 
ulate the end decimal of one or more angles and sides (cf. p. 84). 

By adjusting the observations, as is now possible (Eqs, 12, p. 138), 
the residuals can be inspected individually before any conclusion 
is based on S, the summation of + WyYy^. 

Exercise. Compute the a: and y residuals for the other nme 
paints, and plot them. 

79. The standard error of the calculated ordinates. In accord- 
ance with Eq. 22 on page 167, the standard error of the function 
/(a, 6, c) is 

.,>-."{ 0.000 28(0 + 0.000 084 ( 1 )’ 

- 1 - 0.000 0014 

- 2 X 0.000 0057 £ £ - 2 X 0.000 0084 £ £[ ( 24 ) 


^* + 2 X 0.000 0003 
dcj da do 


In particular, the calculated y with its standard error for this 
problem would be found by writing a-\-bz-^ cx^ for /(o, b, c), the 
result being 

y = 0.2024 -I- 0 . 0457 X + 0 . 0036 ®® 

± ff { 0.00028 + 0.000 0006 ® + 0.000 0626 ®“ 

- 0.000 0168 ®* + 0.000 0014 ®*} 4 ( 25 ) 
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If the factor a in front of the brace be replaced by l.QBj the 
double sign gives the 95 percent error band. This band, laid off 
from the true curve, is expected to embrace 95 percent of the cal- 
culated curves that would be obtained at any abscissa » in a large 
number of experiments like this one. Unfortunately, in practice 
we do not have the true curve, and can only lay off the eiTor band 
above and below the calculated curve, as is done in Fig. 22. The 
band so drawn will vary from one experiment to another (p. 170). 
Moreover, when a- is not known, we can only lay off a “ confidence 
band ” (p. 169), calculated from an estimated value of a (next 
section) . It is only when the number of degrees of freedom reaches 
25 or 30 that the width of the confidence band can be mterpreted 
as an error band, and even then only in randomness. 

80. Calculation of the external estimate of o-. The external 
estimate of v (Sec. 13) is the sum (S) of the weighted squares of 
the residuals, divided by the number of degrees of freedom. Bow 
IV in the solution of the nomal equations gives 

+ w„F/) = 1.68 (26) 

The number of degrees of freedom is 9, this being the number of 
points (12) diminished by the number of parameters (3), whence 

a^iezt) = I = ^ = 0.19 (27) 

Now, in this example, we were furniished with a prior value of o-, 
(0.5; p. 221), and we are thus able to compute which we recall 
is simply S/a^ (p. 15). Wo thus find 

2 S 1.68 
’‘“<72" 0.5® 

= 6.72 (28) 

For 9 degrees of freedom the average value of x® would be 9. The 
value just obtained is less than the average. Fisher’s tables show 

= 0.67 (approx.) 

which interpreted means that, in randomness, in 67 out of 100 
experiments x® would bo greater than 6.72. 
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Example 3. A Formula Useful in Forestry^ 

81. The formula to be fitted. This example serves the purpose 
of illustrating three features: i. the fitting can be done with 
logarithms, the constant or nearly constant characteristic being 
suppressed or nearly suppressed to cut down the number of 
figures required; ii. F is constant throughout; iii. the prior 
value of 0 - can be expressed in terms of some of the parameters, 
so that, finally, the minimized S can be transfoi-med into x®, 
and the fit of the formula judged on this criterion. All three 
features owe their existence both to the form of the fitted function 
and to the experimental material and procedure. One or more 
of them, however, is likely to be encountered in other work. If 

X = the volume of a tree in board feet 

y =s the merchantable height of the tree 

a = its diameter at breast height 


then experience has shown that the equation 

X = ayV 


(29) 


predicts satisfactorily* the values of x from observations on y 

and a. . , x t 

The particular set of data for consideration in the present prob- 
lem consists of 69 points — measurements on the volume, mer- 
chantable height, and diameter, of 66 trees. It will not be neoes- 
saiy to display the full set of points for the discussion mtended 
here; the first six and the last will be sufficient. They come m 
no particular order of size. The logarithms are mitten m J^he 
three right-hand columns, for convenient inspection, smce they 

■will be needed in the fitting. t i • + 

82. Rewriting the function to gain an advantage. Loohng at 
the logarithms in the table of observations, we perceive that the 

1 This problem was furnished by Mr. Jesse H. Buell of the Forest Service, 


^Seis X. Schumacher and F. dos S. Hall, J. Agrio. Be,., vol. 47, 1938: 
pp. 719-734; also Donald Bruce and Francis X. Schumacher, Forest Mensuror 
Urn (McGraw-Hill, 1936), Art. 140. 
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first part ofiithe figures, the "characteristics,” do not vary much- 
in fact, in the Y' and Z' columns the characteristic is unity all 
the way down. What we need to do is to write the formula so 
that the variable part of the logarithms is brought into prominence, 
This can be accomplished by writing the formula as 

»' = a' + by' + ca' (x' = log x, etc.) ( 30 ) 


Data poii Exampli: 3 



Observations ' 

1 

IjOgurltiims 



Volume 

Heielit 

Diam, 




Point 

X 

Y 

Z 

r = 

Y' = 

Z' -I 

No. 

(board 

feet) 

(feet) 

(inches) 

logX 

logy 

logg 

1 

GO 

26 

13.8 

1.778 

1.308 

1.140 

2 

60 

24 

14.0 

1.778 

1.380 

1.146 

3 

120 

20 

la.i 

2.079 

1.462 

1.268 

4 

270 

38 

21.0 

2.431 

1.680 

1.322 

6 

320 

37 

21.6 

2.605 

1.608 

1.334 

e 

180 

30 

16.6 

2.114 

1.477 

1.218 

60 

320 

64 

18,8 

2.606 

1.732 

1.274 




Sums 

162.130 

102.461 

84.000 


and thereupon lowering the characteristics of x', y', and z' by the 
harmless device of subtracting and adding unity to each logarithm, 
arriving finally at the form 

x" = a" + by" + cz" (31) 

where the double primes denote suppressed logarithms, namely, 

x" = x' — 1 = log X — 1 

y" = y' - 1 = log y - 1 

z" = z' - 1 = log z - 1 

o" = o' — l + b + c = loga-l + b + c 


( 32 ) 
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83. Formation and solution of the normal equations. By trans- 
posing the formula all to ouo .xide, wo havi* tho aooeplablo form 

/= j" - («" «”) (33) 


The derivativas of / aro n.s follows: 

fxi = 1| fv' /«' “C 

/((// = -“1) fh “ “jy I ff “S 


Then 




W 


V>x' 

1 


Wzi HV W»' 

-L + 

ix'ii'x ^ y 


(34) 
8, p. 134.) 


=='0.434- - 


-ii'u 2 Wjsl 


( 35 ) 


the last atop coming from KxprfLM*. H«’ tm piigt* 4*3. 

In this investigation, atnl in lelahHl ^•x|)^•ri('n^^ the standard 
errors of .■e, y, and z have Ihm'U fo\uul propurliuuiil, to llm quan- 
tities measured. To lie sjiwifie 

TIkjS.E. <3f / is 7 of X 

“ " " 1J is (5 ” " y 

tt n ill n ** ^ 

If* »l S 

It follows, then, from I5({. 13 on jiago 21 Hint. 



wherefore 

^ = (“7")" J + (n.uf5f3)“ + (aosc)®) (37) 

which is constant tbroughmit, iiul«pi>ntli'Ul of x, ?/, and e. This 
is the second of the two important fwituros tUwtiribed above. 
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ISec. 83] 

Now a is opeu to arbitmiy choice, since weights are not abso- 
lute but are relative only (p. 22 ) ; and a convenient choice ia 
to put 

= 0.4342(0.072 + (0.0Qh)2 + (0.05c)2) ( 33 ) 

whereupon W becmnes unity at all 'points. The value of o-, in this 
problem, is not needed until at the end, when it will be com- 
pared with o(ext). (See Sec. 14, p. 29.) What is more im- 
portant at present, b and c will not be needed for the calculation 
of W, in spite of the fact that a:, y, and s are all subject to error. 
From a computational standpoint, this is a foi-tunate situation, 
resting on the peculiar’ combination of the form of the fitted func- 
tion and the standard errors of x, y, and g. 

As it happens, W being constant (unity) throughout, the same 
results for a, b, and c would come from normal equations set up 
under the (incorrect) assumption that only the measurements 
on volume are subject to error, and that they are of unit weight, 
But estimates of the parameters, however important, are not the 
whole problem; one ought also to consider tho adjustment of the 
observations for a study of tho trends (if any) in the residuals; 
one ought also to know the minimized S for considerations of 
the fit of tho formula, as, for example, by comparing a {ext) with 
the prior a, which fortunately is at hand in this example as it 
was also in the preceding one. If the errors in the diameter and 
merchantable height are masked, none of tho residuals in volume, 
merchantable height, or diameter can be found; moreover, the 
entiy in Row IV of the solution, which should bo S, is instead an 
unknown multiple of it, whei'cfore the possibility of reconciling 
the known experimental conditions with the fit of the curve is 
lost or put on a basis that is likely to do more harm than good. 

Approximate values of a, 6 , and c (hence also of a"), after 
being found by some method or other (see pp. 137 and 138), or 
being known from previous experience, would be used in cal- 
culating the value of 

fo = X" - (Co" + boY" + CoZ") 

at each of the 66 points. The capitals refer to the observed 
values of log a; - 1 , log y — 1 , log a — 1 . 
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Since "PF = 1 throughout, Tables 1 and 2 of Section 60 coalesce, 
and the normal equations are symbolized as follows. A" , B, 
and C are the parameter-residuals. 

Unknoums 


Row 

A" 

B 

C 

1 Sum 

I 

60 

m 

[Z"] 

-Lfol 

2 



[Y"Z"\ 

-[Y'%] 

3 



[Z"Z"] 

-[Z"h\ 

4 




m) 


Since most of the adjustment is already contained in the approxi- 
mate values of a, h, and c, a maximum of two figures would suffice 
in any colunon of X", Y", Z", or /o; and a maximum of three 
figures would likely suffice in the normal equations. Such simpli- 
fication is our compensation for the trouble of computing fo at 
each point. 

The solution would proceed as on page 158. Row IV will con- 
tain the minimized S, correctly distributed among the residuals 
in volume, height, and diameter. The reciprocal matrix found 
in Rows 11) 12, and 13 will contain the variance and product 
variance coefficients of o/ , i, and c. 

As an exercise, the reader might express the variance coefficiente 
of a' and a in terms of cn, cn, etc., found in ffie reciprocal matrix 
in the solution of the normal equations for A , B, and C. 

84. Numerical results. Instead of using approximate values of 
a, b, and c, and computing an/o at each point, Mr. BueU had already 
adopted the somewhat longer process of using oq =bo - cq - 0, 
fo = X', and solving for a', h, and c directly. His normal equations 
are symbolized as follows, directly in terms of the logarithms 
(F' = log etc.). 


Unknowns 


Row 

a! 

h 

c 

= 1 

Sum 

1 

2 

3 

4 

66 

[F'l 

[Y'Y'l 

[Z'] 

lY'Z'] 

[Z'Z'l 

[X'j 

[Y'X'\ 

(Z'X'] 

[X'X'] 

... 
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Numerically, his equations were these: 

Unknowns 


Row 

a' 

b 

C = 1 

I 

00 

102.461000 

84.000000 

152.136000 

2 


16Q.S21326 

131.022337 

237.086322 

3 



107.863544 

196.796661 

4 




366.800622 


The solution was found to be 

o' = -1.78222 = logo, a = 0.01652 
b = 0.87476 
c =2.14226 

Hence the relation found was 

X = 0.0165y°®"82'^* (40) 

Not having at hand the complete form of solution, and in par- 
ticular, not having S as it would appear in the form of solution 
shown in Section 61, page 168, we shall hero make use of Exercise 3 
of Section 61 (see also Exercise 25 of Sec. 70), thus getting 

S = 356.809522 + 152.136000 X 1.78222 
- 237.985322 X 0.87476 
' - 195.795651 X 2.14226 

= 0.324 (41) 

It will be noted that S is here the small remainder left over from 
the addition and subtraction of relatively much larger numbers. 
To secure two figures in S, one must carry a, b, and c through the 
fourth decimal; this is so in spite of the fact that we can not pos- 
sibly rely statistically on so many figures in a, b, and c, a fact that 
would be evident from their standard errors or from forest measure- 
ments in general. This situation is to be contrasted with the 
relatively few figures that would be required for the normal equa- 
tions if good approximate values oq", hoi ‘>>ud cq had been used for 
the calculation of /o at every point; with good approximations, the 
sum [/n/ol would itself be close to the minimized S, so that the 
correction terms need not be carried far. The reader will realize 
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that this matter has been stressed earlier (see, e.g., pp. 163, 175, 
180, 182, and 209). 

We can now make the external estimate of a from the value of S 
computed above, using Eq. 21, page 28, with the result that 


This is to be compared with the prior it®, which from the choice 
made in terms of h and c on page 234 turns out to be 

(T® = 0.434® (0,07® + (0.06 • 0.876)® + (0.05 • 2.142)®} 

= 0.00356 (43) ' 


Thus (r®(e®i) is about 50 percent larger than the prior a®. The 
possibility of this comparison is the third feature mentioned at the 


start. 

A more exact comparison of the two estimates of o- can be mode 
as follows. First of all, we need x*. By the definition of x® on 
page 15, 


^ (T® 0.00366 


(44) 


Since tables of chi-square do not run so high as 63 degrees of free- 
dom, we use Fisher’s function® 

- V2jfc - 1 

which works out to be 2.3, giving P a little over 0.01. This is a 
little low, signifying that it might be well to look carefully at the 
data for inhomogeneities of various kinds. 

It would be interesting to make a study of the residuals as func- 
tions of X, or y, or g, but we shall not stop here except to indicate 
how the residuals would be computed. From Eqs. 10, page 136, 
we have 

X = Z" - (a" + bY" + cZ") (at any point) 

® This remarkable function is written at the bottom of Table III in liber's 
StaUsliaa Methods for Research Workers (Oliver and Boyd), all emtions. 
When k is large, say above 30, it is distributed very nearly as a normal deviate 
with unit standard deviation. 
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whence by Eqs. 12 on page 54 the logarithmic residuals would be 



X 

/0.434 • 0.07\2 

— /l/ — 

— = 

X 

Wx' 


\ 0 ) 

X ^ 

Xb 

/0.434 • 0.06V 

— fj/' ~ 

— — = — 

) U > 

Wyl 

Wyi 

\ J 

-jLf,- 

Xc 

/0.434 • 0.05\2 


= — ■ 

I ) Xc 

Wsi' 

lOj/ 

\ <^ J \ 


(46) 


. dS if , 

Certain special features peculiar to this problem have been 
mentioned, and the remaining details will be omitted; the reader, 
however, will profit from Professor Schumacher’s comments on 
the foregoing. 


86. Comments from Professor Francis X. Schumacher, 

Duke University 

(a) The number of figures required in the solution of Mr. Buell’s 
normal equations could bo cut down by the calculation of an/o at 
every point, as emphasized in Section 84, but perhaps a 'more 
effectual saving of labor would follow upon transfemng the 
origins of coordinates from tho natural zeros to the logarithmic 
means Z'. We know from the first normal equation of 

either of the sets on page 235 that the fitted plane will pass 
through the logarithmic means, which is to say that the final values 
of 0 , b, and c will satisfy 

F= a' + bF + cF 

The transfer of the origins will not only cut down on the number 
of figures required, but will also eliminate the parameter a' and 
reduce the number of normal equations by one, leaving only 6 and 
c as the unknowns, a' to be found afterward by noting that 

a' ^T-bW-cP 

The new sums and cross-products (to be denoted by appending 
the sign “ to the brackets) would be found by making the 
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following reductions from Mr. Buell’s equations: 

[y/y /]0 ^ 159.921325 - 102.451766 = 0.887880 
[V'Z'r = 131.022337 - 102.451 X 84.090/66 = 0.490450 
[Z'Z'Y = 107.853544 - 84.090766 = 0.715240 
[Y'X'\° = 237.985322 - 102.451 X 162.136/66 =« 1.826453 
\Z'X']° = 195.795651 - 84.090 X 152.136/66 = 1.960557 
[X'X']° = 356.809522 - 162.136766 = 6.122213 

Four decimals will suffice, whereupon Mr. Buell’s normal equar 
tions (p. 236) reduce to the following set, which can be solved as 
shown. 


Row 

b 

c = 

1 

Sum 

I 

0.8879 

0.4904 

1.8266 , 

3.2048 

2 


0.7152 

1.9606 

3.1662 

3 

Factoia 


6.1222 

9.9073 

4 

-0.56231 

-0.2708 

-1.0088 

-1.7700 

II 


0.4444 

0.9618 

1.3062V 

5 

-2.05710 


-3.7673 

-6.5926 

6 

-2.14176 


-2.0386 

-2.9903 

ni 


S 

0.3264 

0.3264-^ 

S 


6 

0.8742 


7 


c = 

2.1418 

3,1418V 


The values of b and c just obt^ed agree well enough with those 
on page 236, but with fewer figures and less trouble; and the same 
can be said for the sum of squares 0.3264 seen in Row III. Other- 
wise obtained, 

8 = 6.1222 - 0.8742 X 1.8265 - 2.1418 X 1.9606 
= 0.3263 

affording an interesting check. (The two figures 0.3264 and 0.3263 
for S show a numerical comparison of the two expressions in parts 
c and a respectively of Ehceroise 3 on pp. 163 and 164.) 
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{b) The following suggestion, is offered here in the hope of 
fostering first approximations as a preliminary to the real work of 
fitting by least squares. If the merchantable portion of the tree 
stem were of the same geometrical form in all tree sizes, the volume 
would vary directly with the height and as the square of the 
diameter. Hence useful approximations should be 

b = 1 

c = 2 

a! = X' — Y' ~ 2Z' (not needed in the plan just outlined) 

The problem is then seen as that of finding the effect of changes 
produced by live form of the merchantable solid upon tree volume. 

Example 4. A Sample Suhvey op Canned Goods 

86. Object of the survey. This example is described here, 
because the solution has a wide diversity of application in sample 
surveys; in fact, the solution given here has already been found 
useful in otlier fields. Of couise, each new problem carried with 
it a multitude of theoretical and administrative details that ore 
new and different, and these must be worked out and tailored to 
the new requirements. 

In laying plans for allotments of canned goods for the year 1943, 
the question of current inventories of distributors arose and was 
referred to the Census, with the thought that sampling might be 
introduced to decrease the number of inquiries involved, and the 
expense attached thereto, and — what is more important often- 
times — to decrease the time interval between the collection of 
the data and the completion of the tables. A solution in the form 
of a Bomplp was provided by Messrs. Morris H. Hansen and 
William N. Hurwitz, and was tried out in the Bureau of the 
Census. The country was divided into 24 areas; and within any 
one area, the establishments were divided into classes of five 
different sizes, depending on their inventories of canned goods on 
Date 1. An inventory was taken of the stock in every store on 
Date I, but inventories of only a sample of stores were taken on 
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Fio. 23. Inventories of conned pees on two dates for selected sample stores. 
Each point represents a store. The four lines are drown to show the oslou- 
lated relations for the four different classes. The first class is missing, be- 
cause in this area no store of the first class had oanned peas. 
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Date II, a month later. The sampling scheme diminishes the 
amount of reporting at Date II by requiring reports from all the 
stores in the 5th size class (the highest), but from only half of 



I t 3 4 S 6 

INVENTORY AT EARLIER DATE 

' nCUBAMDB OF UNO 


Fiq, 24. InreatorioB for the fifth class, and the lino through the centroid. 
Each point represents a store. The dashed wedge shows two standard devia- 
tions of the slope, calculated from Eq. 67. 

them in the 4th size class, a quarter of them in the 3d, an eighth of 
them in the 2d, and a sixteenth of them in the 1st or lowest class.'^ 
From the sample, it was possible to make a usable estimate, for 
each area, of the stock that would have been recorded by taking an 

’’ In order to produce reliable estimates of inventories on Date II, by area, 
the sampling ratios were changed for smaller areas, depending on the sizes of 
the stocks on bond. The figures just given constitute a typical set of sampling 
ratios for one of the largest areas. The size class in which a store belongs is 
determined by the number of cans of all kinds of goods on hand— ' pees, 
beans, soup, meat, etc., — but the onalyBis is carried out for each commodity 
separately. This explains why the size classes for peas overlap in Figs. 23 
and 24; a store with a large over-all inventory may be small in poos, and 
vice versa. 
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inventory of all the stores on Date II. Moreover, the design of 
the sample was such that the reliability of these estimates could 
be made on tho basis of some initial trial samples, and sharpened 
by further trials. Some of the underlying theory can be ap- 
proached in terms of curve fitting, and will be presented as such 
here. A forthcoming publication by Messrs. Hansen and Hurwitz 
will contain many other interesting aspects of the problem, paiv 
ticularly from tho sampling angle. 

87. What the sample gives. Let the subscript i denote the size 
class. There being five size classes in any area, i will run through 
the values 1, 2, 3, 4, 5. Let the subscript j refer to a particu- 
lar store in the ith class. Then j will run through the values 
1 , 2 , ■ • •, m. 

rii is the number of establishments sampled in the 
ith class 

Ni is the total number of establishments in that 
class 

Xij is the inventory (number of cases of peas® on 
hand) in the jth establishment of the ith class 
on Date I (Imown for every store) 

Zij is the inventory of this same store on Date II 
(known only for the stores that are in the 
sample) 

N‘ 

Xi= Xii = the complete inventory of all stores m the ith 
on Date I. (In this summation, j runs 
from 1 to Ni, to indude all the stores in the 
class. Xi is known.) 


:i = L = the inventory of just the sample stores in the 
ith stratum, on Date 1. (In this summation, 
j runs from 1 only to w,-, since only the sample 
retailers are admitted in this sum. x, is 
known.) 


8 For oonvenience, the analysis wiU be carried through with reference to 
canned peas, though obviously any other commodity or commodity group 
could be substituted. 
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Ni 

H Sij, similar to Xi, except that this is for Date II. 
{Zi is to be estimated.) 

rn 

Zi = £ Sij, similar to Xi, except that this is for Date II. 
(a,- is known.) 

5 

Z = £ Zi = the established inventory on Date II for the 
sum of all the five classes in this area. 

88. The estimated inventory and its standard error. One way 
to estimate the inventory for class i on Date II is to say that it is 
proportional to the inventory of that class on Date I, in accordance 
with which we write 

Zi = biXi (40) 

Then 

s 

Z = 2 Zi = Zx + Za + Z 3 + Z 4 + Zb (47) 

iail 

is the estimated total inventory of all five classes in the area on 
Date II. Curve fitting enters the problem in the determination of 
usable values of h from the sample of stores in each class, and in 
the calculation of the variances of the estimated inventories. 

The assumption will bo that, except for accidental influences, 
such as weather, delayed shipping schedules, and mistakes in 
counting, all the stores of size class i would increase or decrease 
about the same relative amount between the two dates. This 
assumption, in this problem, has been found to lead to useful 
results. ' Of course, outside this particular field, or under other 
conditions, the same assumptions might lead to difficulty. It 
is only by careful investigation that one is able to say in advance 
under what conditions his assumptions will lead to usable predic- 
tions. Of course, the assumption that the inventories are each 
about the same on the two dates will be found violated by many 
individual stores, but on the whole it will be close enough for the 
purpose intended. 

In evaluating the error in the slope hi, we recognize the ejdstence 
of accidental influence of variation in both Xfy and Zy. The bigger 
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the inventory, the bigger the standard error of the accidental 
variations. Hence we shall put 


Wx 


» 





(48) 

(49) 


wherein j*, as usual, is the standard error of observations of unit 
weight. We might write similar equations for the weight and 
standard error of ay (inventory of a sample store on Date II), but, 
if the two inventories ajy and ay are not greatly different, it 
will be sufficient to make the x and z weights equal, thereby writing 


w»., = Wx., = — (50) 

and 

= (51) 

The standard error resulting from the accidental influences on 
Xi on Date I can be found as follows: 

Hi 

Xi=^xii (62) 

3=1 


Hence by the result obtained in Exercise 2 on page 42, 

= (®il + ®t2 -I (^2) 

We here take 

F szij — hfCij 

Then 

L = 


FxFx , F.Fx 


Wx W, 

_ ^ ^ I _ l + bi^ 

~ Wx~ Wt Wij 


(54) 

(at point ij ; Eq. 8, p. 134) 
( 66 ) 


Wij is here written for the weight of either ®y or ay. 
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The normal equation for 6,- is shown below. The subscript ij 
on M), X, and z is omitted for convenience, and the sums (2) are 
taken over the sample stores (i.e., j runs from 1 to n,', while i 
remains constant). 


hi 

= 1 

C 

E K)®* 

E ws* 

1 


l+bi^ 



The resulting solution for the slope is 


6i = 


"^wxz 
S Zjj _ ^ 

E 


(Compare with Eq. 34 on p. 31.) 


( 66 ) 


since to,, /By is to be counted equal to unity (Eq. 50) and *,■ is 
the inventory of the m sample stores on Date I. Note that by 
the value of just obtained, the line is to be drawn from the 
origin to the centroid of the n,- points. The variance of hi is seen 
from the normal equation to be 




'^wx‘ 




Now for the estimated inventory on Date II we recall that 

2= i.Zi= ZhXi (47) 


(-1 




whereupon, by the result obtained in Exercise 3 on page 43, it 
follows that 

The first term in the brackets arises from the sampling error in 
the slope h{, which will vary from one set of sample stores to 
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another. The second term arises from the fluotuationa to be 
expected in the complete inventory Xi at Date I. 

It remains to estimate a^. It is best to do this for each nlaas 
separately. Since both sy and are assumed to be subject to 
the influence of chance fluctuations, we measure the residuals from 
each point perpendicularly to the line s = hue, and write 

= S (Cf. Eq. 21 , p. 28) (59) 

i=l Wi — 1 


^__resj£_ 
~ 1 ) 


(60) 


The factor nj - 1 arises from the reduction of by unity for the 
single parameter hi. 

Perhaps the simplest ^ray to evaluate the sum of the weighted 
squares of the residuals (the summation over j called for in the 
formula just written) is to measure each residual graphically, 
square it, and divide by the value of xa as read at the foot of 
the perpendioular dropped from the observed point to the line. 

Another but theoretitsally less exact method of evaluating 
this summation would be simply to calculate the lesiduals from 
the line by Ihe formula 

Residual = zy - bfca (61) 

as if they were measured in the vertical. The sum of the 
weighted squares calculated with vertical deviations will be 
about half the sum of the waited squares calculated with 
perpendicular distances, and the factor 2 can be applied to 
compensate. 

It will be sufficient for the purpose to set b,- in the brackets of 
Eq. 58 equal to unity, an assumption already made in the wrighte, 
and justified by the slopes in Fig. 23, Wili this simplification it 
is found that 

The five terms called for in the summation over i on the right- 
hand side are the five separate values of 
inventories Zi, Z 2 , Zz, Z 4 , Zg. 
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Fig. 23 shows a plot of the points for canned peas for the stores 
in the 2d, 3d, 4th, and 5th classes, in the state of New York. 
For this particular commodity, there was no store in the 1st class. 
The units of measurement are designated by the scales. The 
slopes of the four lines are ba, h, h, and bs- The scatter of the 
points is more than one might hope for, but the method gives 
useful results nevertheless. 

Fig. 24 shows the points for the 5th class separately, with a 
wedge laid off each si^ of the line to show the width of two esti- 
mated standard errors. This wedge is indistinguishable from the 
95 percent confidence band. 

89. Summaiy of the enors to be considered; effect on sample 
designs.^ There are two kinds of problems that arise in sampling 
inventories, and we might designate them as Problem A and Prob- 
lem B. Problem A consists simply of sampling a pile of schedules. 
Every store in an area (e.g.. New York state) has presumably 
sent in an itemized schedule showing the number of cans of peas 
and other commodities on hand at a certain date. In the discus- 
sion that now confronts us, this date was Date II, but this is 
unimportant so far as the description of Problem A is concerned. 
The question is how to find, by sampling, a number (an estimate) 
that for purposes of action can be used in place of the total inven- 
tory of peas contained in the entire pile of schedules. This is the 
problem, regardless of whether the responses written on the 
schedules are correct or not; and the error in the sample estimate 
will be the difference between that estimate and the actual count 
contained on the schedules, whether it be right or wrong. 

The number of stores is large, perhaps in the thousands. The 
reason for taking the sample would be to hasten the processing^® 
of the data, and to get it done for less money. The pile of inven- 
tories might be so big, SiUd the deadline so short, that there is time 

° The author is exceedingly indebted to Messrs. Hansen and liurwitz, not 
only for permission to use their example, but more especially for assistance 
rendered in numerous discussions, during which the recognition and evaluation 
of the five different sources of errors were evolved. 

The term “ processing of data ’’ refers to olEco operations in the nature 
of editing, coding, transcribing, punching and tabulating, posting and con- 
solidating, in the production of final tables or summaries. 
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to work with a sample, but not with the complete count. So far 
as Problem A is concerned here, there are two mutually exclusive 
sources of error, which may be outlined below. 


i. The stores that are drawn into the sample are designated 
as the sample stores. In one particular sample survey these are 
a particular set of stores. But if the sample were redrawn from 
the same universe of stores, there would be a different set of 
sample stores, and another estimate Z of the total count. It 
follows that there is a sampling error in the estimated total 
count, and a sampling error in the calculated variances, arising 
from the sdeotion of sample stores. 

Messrs. Hansen and Hurwitz have made an approximate 
evaluation of this source of error, and their result is 




<»1 Ni “ 1 Jli 


S ( ZiY 


(63) 


If the factor (1V< - ni)/{Nt - 1) is replaced by unity, this 
error is seen to be about half the fourth source of error men- 
tioned below. This factor, incidentally, reduces the first source 
of error to zero when aU the schedules of a class are processed, 
for then IVi — Ji< = 0. 

ii. The &Bt source of error can be decreased by using an ap- 
proximate relationship between x and provided one exists. 
An assumption is useful if it malces useful predictions. If some 
other set of assumptions turns out to be better for purposes of 
prediction, and if the extra cost involved in ofiioe procedure is 
not too great, a change might be warranted. A change in the 
assumption of a relationship will produce different results, not 
only in the estimated total inventories on Date II, but also in 
the estimated variance of that total inventory. 

Sources i and ii do not both exist simultaneously. Messrs. 
TTaTiaRn and Hurwitz, in their evaluation of the sampling error 
(mentioned above), did not make use of any assumed relation- 
ship: hence their formula applies to source i only. There is no 
way of evaluating the second source of error analytically, even 

when it exists. , , 

The effect of either or both of these two sources of error con be 
reduced to any desired degree by taking a big enough sample. 
Messrs. Hansen and Hurwitz wished particularly to reduce 
the error in the fifth class (the largest invmtoriffl); hence 
they took all of it. A 10 percent error in the fifth class would 
amount to as many cons of peas as a 60 percent error m one of the 
lower classes. 
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EXERCISES AND NOTES 


ISec. 891 


Problem B includes some other aspects that need to be con- 
sidered when one takes into account the influences that aflect the 
figures recorded on the schedules. In this problem, the inven- 
tories for Date II would ordinarily be collected on a sample basis, 
and the reports that have been received up to a certain deadline 
date would be processed. The action that is to be taken (policies 
in distribution) will affect all the stores in the area, those in the 
sample, and those not. A number of sources of error must be 
considered. 

iii. Late reports introduce an error. It would of course be 
dangerous, if not folly, to assume that the late reports are a ran- 
dom sample of the universe. No attempt is made here to eval- 
uate the bias arising from late reports. 

In a sampling project, the total number of reporting stores 
may be so small that individual attention oan be given to them, 
to reduce the proportion of late reports and the uncertainty 
introduced by them. For instance, one might send out tele- 
grams just before the deadline to bring in some of the reports 
that threaten to be delinquent. Moreover, one might subse- 
quently follow up some of the late reports, to decide, on the basis 
of empirical evidence, which way and how much the late reports 
affeot the estimates. ' 

iv. There are random errors in the responses of the sample 
stores, and there are fiuctuations in their inventories owing to 
extraneous natural influences (such as the weather and freight 
tie-ups in and out), all of which throw the points away from 
whatever relationship may otherwise exist between the invon- 
tories on the two dates. 

This source of error is the first term in the brackets of Eq. 68, 
end it is seen to be smaller os increases, which is to say that the 
fourth source of error grows smaller as Ihe sample grows bigger. 

This source of error, unlike the first and second, con not be 
reduced to zero by taking all the stores in any class. 

The 2-8igma band in Fig. 24 is calculated from Eq. 57 and 
corresponds to the first term in the brackets of Eq. 62. This 
band shows how large a sample must be taken to reduce the 
fourth source of error to some desired degree. An assumed 
relationship other than the one adopted would lead to another 
band. 

Deliberate errors of reporting are usually not random, and 
constitute an insidious problem of another kind. It is conceiva^- 
ble that under-reporting oanoola out, os when JTi and X{ are both 
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reported as just 50 percent of their true values; h would be 
twice as big, but Zi would be unaSeoted. 

V. Random errors of response, and extraneous natural influ- 
ences are present, not only in the sample stores, but in the 
reported inventories of all the stores. As a consequence, the 
total inventory Xi of any class is affected. The effect of this 
error on the total inventory Zi is evaluated by the second term 
in the brackets of Eq. 58. 

vi. There is an error in Problem B arising from the assump- 
tion of a particular relationship and weighting. This corre- 
spends to the second source of error, mentioned under Problem 
A, Again, there is no way of evaluating this source analytically. 
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Tables fob Masing Random Obsebvations foe Class 
Illustbation 


numoer represents one observation. The numbers may be 
taken ou* m any order - across, cornerwise, or in any systematic 
fashion that does not make use of the size of the number. 


Pabt A : Nobmal Deviates Dibectly in Units of the 
Standabd Ebeob 

(Thi. table eomea from a paper by Edward L, Dodd, Bofefe 
MatemUco, Bumos Aires, Ano xv, 1942: pp. 76-7, with the land 
permission of the author and editor. These numbers were ob- 

“‘‘‘T « two I»«ee d Tippett-. 


- 0.64 

- 0.21 

- 0.80 

- 1.69 

- 0.30 

' 0.22 
1.46 
- 0.84 
1.72 
0.03 

0.17 

- 1.27 

0.48 

- 0.83 

0.14 


- 0.12 

- 0.79 

0.50 

0.93 

0.81 

1.64 

- 0.40 

1.34 

1.42 

0.77 


0.42 

1.67 

0,67 

0,06 

1.37 

- 0,26 

- 1.02 

- 0,41 

- 1.22 

- 1.08 

2.04 

- 1.08 

- 0.60 

0.28 

1,30 

0.83 

0.68 

- 0,37 

0.26 

0.63 

0.23 

0.09 

- 1.28 

0.80 

0.28 

- 0.48 

- 1.13 

0.60 

- 0.19 

1.18 

- 0.67 

- 0.80 

0.70 

- 0.67 

- 1,00 

0.00 

- 0,41 

- 0,96 

1.68 

- 0,87 

- 0.07 

0,11 

0.63 

- 0,34 

0.01 

- 0,66 

1.14 

- 1.72 

- 0.04 

- 0.41 

0.30 

0.20 

1.01 

1.19 

0.66 

0.27 

0.01 

1.16 

- 0.80 

- 0,26 

- 0.38 

0.49 

0.03 

- 0.77 

- 0.06 

1.45 

- 1.17 

0,17 

- 0,03 

- 0.64 

0.33 

0.40 

- 0.31 

0.63 

0.07 

0.36 

- 0.77 

0.63 

- 0.66 

- 0.89 

0.01 

0.00 

- 0.40 

- 1.02 

0.38 

0.81 

- 1.46 

0.97 

0.24 

0.10 

- 0.91 

0.38 

- 0.22 

- 0,22 

- 0,33 

- 0.44 

0.17 

0.63 

- 0,22 

0.24 

- 0.28 

1.48 

0.48 

1.66 

1.98 

1.63 

0.73 

- 1.19 

0.17 

- 0.60 

- 0.19 

- 0.97 

0.69 

- 0.27 

- 0.20 

- 0.67 

0.11 

- 1.14 

0.24 

0.00 

0.66 

0.33 

- 0.98 

1.47 

- 0.82 

- 0.40 

0.26 

0.02 

- 0.68 

- 0.66 

- 1.46 

1.64 

- 1.60 

0.66 

0.72 

- 0,64 

- 0.43 

0.39 

- 0,24 

- 2.30 

- 1.27 

- 1.24 

- 1.33 

- 0,82 

0.63 

0.36 

- 1.06 

- 0,10 

0.33 

- 1,12 



262 




o.ie 

- 0.01 

0.74 

1.16 

0.37 


- 0.69 

- 0.02 

0.56 

0.30 

0.57 

0.70 

- 0.70 

0.37 

0.12 

1.72 

2.62 

0.73 

0.02 

- 0.44 

1.11 


- 0.13 

- 0.06 

- 0,18 

- 0.66 

- 0,86 
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1.06 

1.67 

- 1.19 

1.87 

1.41 

- 0.37 

0.63 

1.17 

0.25 

- 2,92 

1.63 

— 0.61 

1.72 

- 0.08 

1 29 

- 0.90 

- 1.76 

n 1 E 

- 2.27 

0.44 

- 0.88 

0.14 

- 1.47 

0.35 

- 0.26 

- 0.26 

- 0.24 

- 0.31 

- 1.02 

- 0.78 

-oioo 

0.91 

U » J .0 
- 1.09 
- 0.12 

— 0.72 

O.OQ 

0.34 

0.89 

- 0.14 

- 0.39 

- 1.07 

0.64 

- 0.40 

0.67 

1,27 

- 0.69 

- 0.34 

0.63 

- 1.38 

- 0.98 

0.17 

1.31 

- 1.62 

0.70 

1.21 

- 0.40 
0.44 
n nd . 

- 0.14 

- 0.92 

- 0.60 

- 0.48 

0.81 

1.10 

- 0.02 

- 0.38 

0.19 

1.27 

0.30 

— 1.42 

0.40 

0.36 

U.Urc 

1.08 

- 2.76 

1,01 

- 0.41 

- 0.04 

1.36 

1.33 

0.26 

- 0.07 

- 2.01 

- 0.62 

0 . 07 ' 

0.05 

0.01 

- 0.70 

1.04 

- 2.01 

- 0.01 

1.16 

2.08 

0.00 

0.81 

- 0.77 

- 3.89 

0.42 

1.68 

- 0.72 

1.64 

- 0.90 

1.03 

- 0.67 

- 0.01 

- 1.01 

- 0.70 

- 0.97 

- 1.46 

- 0.02 

- 0.87 

- 0.36 

1.38 

- 1.24 

- 0.68 

- 0,92 

0.06 

- 1.08 

- 0.49 

- 0.20 

- 0.81 

- 1.18 

- 0.28 

- 1.23 

-O.flfl 

- 0.08 

- 0.70 

- 0.68 

0.78 

- 0.90 

1.78 

0.36 

- 0.74 

- 2.47 

1.36 

- 0.33 

- 0.32 

- 0.48 

0.01 

2.22 

- 0.22 

— 1.11 

0.02 

- 0.36 

HA 

- 0.91 

0.76 

0.16 

0.67 

1.06 

- 1.99 

0 77 

AtUO 

n 1ft 

0.31 

1.76 

1.78 

- 0.80 

0.76 

- 0.81 

o!oi 

Ui lu 

- 1.69 

- 0.00 

- 0.12 

- 0.00 

0.86 

- 0.09 

- 1.18 

0.61 

0.97 

- 1.00 

0.62 

- 0.49 

0.01 

- 0.03 

- 1.01 

2.10 

o !47 

- 0.01 

- 0.62 

- 1.06 

- 0.68 

- 1.06 

- 1.30 

0.61 

— oisi 

- 0.02 

2.02 

0.00 

- 0.26 

- 0.60 

- 1.60 

— 0.26 

0 88 

- 0.72 

- 1.00 

0.67 

— 1.96 

- 0.76 

- 0.23 

- 0.47 

2 ! 07 

- 1.16 

- 0.67 

0.03 

- 0.26 

- 0.34 

1.03 

— 0.04 

0.12 

0.11 

- 0.02 

- 1.38 

0.60 

- 0.06 

- 1.12 

1.13 

— 0.77 

- 0.46 

- 0.73 

- 0.22 

2.26 

- 0.06 

- 0.12 

— 0.94 

— 0.66 

0.78 

- 1.88 

1.20 

- 0.70 

0.77 

- 1.44 

- 1.10 

0.68 

0.40 

- 0.20 

- 0.30 

0.02 

- 0.13 

- 0,60 

- 1.46 

- 0.01 

1.30 

- 0.30 

1.37 

- 0.80 

0.46 

1.81 

0.88 

- 0.49 

0.08 

- 1.20 

- 1,63 

0.38 

- 0.04 

1.13 

1.19 

1.07 

- 2.24 

- 0.00 

0.86 

1.42 

- 0.80 

- 0.76 

0.01 

1.10 

- 0.72 

0,70 

1.00 

0.68 

1.00 

- 0.26 

- 1.34 

0.07 

- 1.80 

0,07 

- 0,26 

- 1.94 

1.44 

0.20 

0.18 

- 2.72 

0.70 

1.89 

0.07 

- 0.19 

2.26 

- 0.02 

- 1.29 

1.36 

- 0.07 

- 0.19 

- 1.02 

- 2.01 

- 2.28 

0.39 

1.11 

- 0.07 

0.42 

- 0,87 

- 0,40 

1.37 

0,29 

0.23 

- 1.46 

- 0.28 

0.40 

- 1.00 

1.36 

- 0,06 

2.34 

- 0.31 

0.04 

- 0.60 

- 0.12 

0.84 

1.10 

- 0.42 

0.82 

- 0.04 

- 0,17 

1.79 

0.18 

2.27 

- 0.60 

- 1.34 

- 0.22 

0.43 

- 1.41 

0.29 

- 0.21 

- 1.00 

- 0.60 

- 1.46 

- 1.13 

1.28 

- 1.02 

- 1.06 

- 0.12 

- 0.08 

- 0.30 

0.08 

0.72 

0.66 

- 1,27 

- 0.70 

- 0.00 

- 1.64 

2.10 

— 0.14 

0.17 

0.73 

- 1.18 

- 0.73 

0.44 

- 0.37 

2.12 

1.40 

- 1.64 

0.38 

0.09 

1.26 

0.30 

- 1.84 

0.81 

0.92 

- 1.37 

0.87 

0.18 

- 0.80 

- 0.07 

- 0.18 

- 0.80 

0.08 

- 0.38 

1.27 

0.66 

- 0.30 

- 1.12 

- 1.17 

- 0.43 

1.22 

- 0.27 

0.46 

- 0.96 

- 1.85 

0.01 

— 0.26 

- 0.81 

2.08 

1.10 

- 0.44 

0.68 

0.20 

0.79 

- 0.83 

0.04 

- 0.84 

- 0.64 

- 1.98 

0.73 

- 0.67 

- 0.86 

1.28 

- 1.27 

0.00 

0.74 

- 0.21 

0.89 

0.60 

2.64 

— 1.38 

0.60 

- 0,27 

0.84 

- 1.37 

0.20 

- 0.13 

— 1.84 

2.38 

0.00 

0.77 

0.61 

- 1.30 

- 0.86 

2.26 

2.00 

0.81 

- 0.22 

0.48 

- 1.99 

- 1.11 

0.66 

1.20 

- 0.90 

0.79 

1.06 

- 0.23 

- 1.26 

- 0.63 

0.27 

- 1.30 
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- 0.81 

- 1 . C3 

- 1.02 

0.64 

- 0.44 

0.40 

1.27 

1.41 

1.06 

- 0.78 

0.50 

- 0.42 

2.00 

1.52 

0,19 

0.48 

0.42 

- 0.63 

0.48 

- 0.12 


- 1,10 

- 0.31 

0.32 

- 2.00 

0.24 

0,47 

- 0.24 

1.37 

0.35 

0.64 

0.40 

- 1.01 

- 0.32 

0.80 

0..62 

- 0.00 

0.07 

1,75 

- 0.40 

- 0.36 

- 1.78 

- 0,72 

- 1.70 

- 0.47 

0.84 

0,55 

2.04 

- 0.80 

- 0,66 

0.18 

- 0.60 

- 0.81 

- 0.43 

- 0,37 

- 1.78 

0.24 

0.14 

- 0,07 

0.13 

0.16 

0.12 

- 2.07 

0.33 

- 0.37 

0.31 

0.80 

- 0.14 

- 0,54 

0.46 

- 0.02 

- 2.48 

- 0.10 

1.16 

1.01 

- 0.03 

1.67 

- 0,38 

0.76 

- 0.28 

0.60 


- 1.53 

- 1.48 

- 0.75 

- 1.08 

0.71 

- 0,02 

- 0.81 

0.96 

- 0.30 

- 0.21 

- 1,72 

- 0.39 

0.80 

0.60 

- 0,10 

1.33 

- 0.61 

- 0.01 

- 1.26 

0.34 

1.27 

- 0.00 

0.09 

- 0.16 

- 1.02 

0.80 

- 0.13 

- 1.01 

0.70 

- 0.02 

- 0.14 

1.07 

- 1.84 

- 0.27 

0.00 

1.07 

- 0.32 

- 0.48 

- 0,34 

0.01 

0.62 

- 0.27 

- 0.52 

0.30 

1.02 

2.55 

1.80 

- 0.36 

- 1.76 

1.23 

- 0.06 

- 0.17 

1.28 

- 1.16 

0.40 

0.74 

2.62 

- 1,11 

0.50 

- 0.03 


0,07 

- 0.08 

0.30 

- 0.32 

1.74 


- 1.00 

- 0.20 

- 2.08 

- 0.88 

2.41 

1.80 

0.22 

- 0.15 

0.60 

- 0.83 

o.oe 

0.64 

- 1.00 

1.02 

0.42 
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Part J3: Normal Distribtition oi? the Numbers ebom 0000 
TO 9999. Class Interval .2(7. 

(This table is to be used in conjunction with Tippett’s numbers, 
in circumstances where a longer series than that in Part A is 
required, or where it is desired to use pages of Tippett’s numbers 
other than the first two.) 


Interval Cumulative 

Center Limits area 


Area of 
interv^ 


Intervals for Center 
Tippett's numbers of 
(0000-9999) interval 


— M 0 


- 3 . 8 o - 

— 3 . 7 a ' 

0.000 

1078 

0.000 1078 

0000 

- 3 . 8 a 

- 3.0 

- 3.6 

.000 

2326 

.000 1248 

0001 

- 3.6 

- 3.4 

- 3.3 

.000 

4834 

.000 2608 

0002-0004 

- 3.4 

- 3.2 

- 3.1 

.000 

9676 

.000 4842 

0006-0009 

- 3,2 

- 3.0 

- 2.9 

.001 

8668 

.000 8982 

0010-0018 

- 3.0 

- 2.8 

- 2.7 

.003 

4670 

.001 6012 

0019-0034 

- 2.8 

- 2.6 

- 2.5 

.006 

2097 

.002 7427 

0035-0061 

- 2.6 

- 2.4 

- 2.3 

.010 

7241 

.004 6144 

0062-0106 

- 2.4 

- 2.2 

- 2.1 

.017 

8644 

.007 1403 

0107-0178 

- 2.2 

- 2.0 

- 1.9 

.028 

7186 

.010 8622 

0179-0286 

- 2.0 

- 1.8 

- 1.7 

.044 

6666 

.016 8489 

0287-0446 

- 1.8 

- 1.6 

- 1.6 

.006 

8072 

.022 2417 

0440-0667 

- 1.6 

- 1.4 

- 1.3 

.096 

8006 

.029 9933 

0668-0967 

- 1.4 

- 1.2 

- 1.1 

.136 

6081 

.038 8666 

0068-1366 

- 1,2 

- 1.0 

- 0.9 

.184 

0001 

.048 3940 

1357-1840 

- 1.0 

- 0.8 

- 0.7 

.241 

9637 

.067 9036 

1841-2419 

- 0.8 

- 0.0 

- 0.6 

.308 

6376 

.066 6738 

2420-3084 

- 0.6 

- 0.4 

- 0.3 

.382 

0886 

.073 6511 

3086-3820 

- 0.4 

- 0.2 

- 0.1 

.460 

1722 

.078 0836 

3821-4601 

- 0.2 

0 

0.1 

.639 

8278 

.079 6666 

4602-6397 

0 

0.2 

0.3 

.617 

9114 

.078 0830 

6398-6178 

0.2 

0.4 

0.6 

.691 

4626 

.073 6611 

6179-6914 

0.4 

0.6 

0.7 

.768 

0363 

.006 6738 

6916-7679 

0.6 

0.8 

0.9 

.816 

9399 

.067 9036 

7680-8168 

0.8 

1.0 

1.1 

.864 

3339 

.048 3940 

8169-8642 

1.0 

1.2 

1.3 

.903 

1906 

.038 8666 

8643-9031 

1.2 

1.4 

1.6 

.933 

1928 

.029 9933 

9032-9331 

1.4 

1.6 

1.7 

.966 

4346 

.022 2417 

9332-9563 

1.6 

1.8 

1.9 

.971 

2834 

.016 8489 

9654-9712 

1.8 

2.0 

2.1 

.982 

1366 

.010 8522 

9713-9820 

2.0 

2.2 

2.4 

2.6 

2.3 

2.6 

2.7 

.989 

.993 

.996 

2769 

7903 

6330 

.007 1403 
.004 6144 
.002 7427 

9821-9892 

9803-9937 

9038-9904 

2.2 

2.4 

2.6 

2.8 

2.9 

.998 

1342 

.001 6012 

0966-9980 

2.8 

3.0 

3.1 

.999 

0324 

.000 8982 

9931-9980 

3.0 

3.2 

3.4 

3.3 

3.6 

.099 

.999 

6166 

7674 

. 000 4842 
.000 2508 

9990-9994 

9996-9997 

3.2 

3.4 

3.0 

3.7 

.999 

8922 

.000 1248 

9998 

3.6 

3.8 

09 

1 

,000 1078 

9699 

3.8 
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Adjusted observations, 2, 16; weights 
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167 

Adjusting sample frequencies, 98 ff.; 
by iterative proportions, 115; by 
the Bruyfere method, 124; by the 
Stephan method, 121; when only 
one cell requires adjustment, 
119 

Adjustment, formulas for, 52, 74, 
138; geometry of, 132, 133, 142, 
144, 228; nature of, 13; numerical 
illustration, 81, 227; plane triangle, 
7, 00, 74; procedure, 6, 139, 140; 
segments of a line, 8, 86 
Adjustment for bias, 10, 127 
Adjustment of parameters, formulas 
for, 62, 136; second, 62, 180 
A. 0. Aitkbn, 16, 19, 100, 173 
Alternate hypothesis, 28 
Analysis of variance, 27, 29 
R. L. Andehson, 173 
Approximate values of parameters, 
61, 62, 137, 163; method of aver- 
ages, 137; method of selected 
points, 138; method of zero sum, 
137 

Auxiliary constants, 91, 94; number 
of, 06 

Averages, method of, 137 
Bessep, 27, 60 

Bias, adjustment for, 10, 127; detec- 
' tion of, 11 
M. D. Binoham, 169 
Raymond T. Bisoe, 27, 29, 88, 169, 
173, 176 


Proper names are in capitals.) 

0. I. Buss, 210 

MaXIME BdCHER, 68 

Donald Bbucb, 231 

Paul T. BRUiiiBB, method, 123 ff. 

Jesse H. Buell, 231, 238 

Caloulated curve, 10, 18, 132, 133 
Calculated points, 18, 130, 132, 133; 
possible and impossible positions, 
146 

Calculation of mean and standard 
deviation, ISO; rapid method of, 
161 

Norman Campbell, 137 
Cauchy distribution, 39 
Cauchy method for obtaining ap- 
projdmations of parameters, 138 
Cell, definition of, 98 
Cdl frequency, 98; adjustment of, 
99; estimation of, 101 
Centroid, 174, 181; see also Quasi 
center 

Chio’s pivotal expansion (of deter- 
minants), 161 

Chi-square, definition of, 16, 22, 27, 
88; distribution of, 141 
Chi-test, 15, 18, 132 
Closure, error of, 84, 109, 228 
Condition equations, 60, 61, 132; 
derivatives of, 61; geometric, 
69 ff., 70, 76; numerical values of 
derivatives, 71; reduced, 53; with- 
out parameters, 69 
Confidence bands, 169, 170, 171, 
230; see also Error bands 
Consistency; see External consist- 
ency and Internal consistenoy 
Controls, 100 
267 
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Cornor check (sum check), 72, 156 
Correlates, or correlatives, 63 
Correlation coefficient, 177 
Covariance (product variance), 10, 
160 

Curve; calculated, 18, 132, 133; 
dosage-mortality, 210; true, 132, 
133 

Curve fitting; an isotherm, 212; 
grapliical conaidcrations, 130, ld3; 
miscellaneous fuuctians, 208; poly- 
nomial, 172, 218; the exponen- 
tial and its logarithmic form, 191; 
the exponential with a linear com- 
ponent, 203; the hyperbola and 
its logarithmic form, 204; the 
hyperbola with a linear component, 
206; the line, 173; the normal 
equations, 136; the parabola, 187; 
the purpose, 128 
Emanuel Czobeb, 142 

Data, object of taking, 1 
Datum, arbitrni'y, 161 
Hahold T. Davis, 173 
Degrees of freedom, 18, 34, 141 
Deviates, normal, 210, 262 
Edwaed L. Dodd, 262 
M. H, Doolittle, 66, 167 
Dosage-mortoJity curve, 210 

J. F. Enckb, 60 

Equations of condition; see Con- 
dition equations 

Error, in transformation to loga- 
rithms, 44; of closure, 84, 109, 228 
Error bands, 168, 171, 230; see also 
Confidence bands 

Errors of sampling; see Sampling 
errors 

Estimate of <r, by external consist- 
ency, 27, 28, 34, 230; by internal 
consistency, 20; unbiased, 27, 168 


Exponential; logarltlimic form, 191; 

with linear component, 203 
External consistency, 27, 28; see ako 
Internal consistency 
Extci'nol estimate of a compared 
with prior a, 230 
Moiidecai Ezekiel, 138 

Face totals, 112 

R. A. Fwiiee, 18, 22, 28, 30, 34, 
168, 169, 171, 172, 209, 218, 230, 
237 

Forest mensuration (example), 231 
IjiisTEit R. Fiiankel, 32 
Freezing (near indeterminacy of 
equations), 160 

Frequency (cell frequency), 08 

W. L. Gaines, 198, 207 
G. R. Gause, 198 
Gauss, 15, 10, 27, 63, 64, 66, 68, 60, 
00. 90, 140, 141, 167, 169, 160, 
167, 180 

Gauss brackets, 54 
Gauss symbols, 167 
General normal equations, 65 
Geometric conditions, 69, 70, 76; 
solution without Lagrange multi- 
pliers, 62 

Conn ADO Gini, 184 
M. A. CtinsHicK, 169 
Goodness of fit, 18 

Graphical considerations of curve 
fitting, 130, 143 

F. DOS S. Hall, 231 
Monms H. Hansen, 47, 240, 243, 
248, 249 

Hatchability of eggs, 32 
J. F. Hatboud, 66 
Helmeht, 142 
Hahold Hotelling, 159 
E. E. Houseman, 173 
David Hume, 12 
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'WusLim N. Ht7rwitz, '240, 243, 248, 
249 

Hyperbola, generalized, 204; loga- 
Tithmic form, 205; with, linear 
component, 206 
Hypothesie, alternate, 28 

Indeterminacy (near indeterminacy), 
160, 161, 163 

Information (Fisher), amount of, 22 
Instability, 160, 217 
Internal consistency, 20; see dso 
External consistency 
Inverse matrix; see Reciprocal matrix 
Isotherm, fitting of, 212 
Iterative proportions; method, 116; 
simplification when only one cell 
requires adjustment, 110; three 
dimensions, 117; two dimensions, 
116 

Jacobi, 159 
Jacobian, 160 

Thtman L. KELLmr, 169 
J. M. Edynss, 12 
Cbaboib H. KummeiiI/, 64, 140, 145, 
184 

L coefficients, 66, 69, 134, 136 
Laobaiiqb, 63 

Lagrange multipliers; calculations 
of, 91, 92, 93, 104, 109, 110, 136; 
method of, 53; normal equations 
for, 82, 86, 109, 111; number of, 
66,96; solution without Lagrange 
, multipliers, 62 

Least squares; computation for 
fitting curves, 168; formulation of 
general problem, 49 ff. ; method 
of, 15; principle of, 2, 14-Ifl 
0. M. Lelahd, 64 
G. J. Lidstonb, 27 
Line segment, 8, 86 
Line of worst fit, 184 


Logarithmic form; generalized hy- 
perbola, 204; (rf the exponentid, 
191, 193; special remarks, 196, 
198, 201 

Marginal total (rim total), 08 
Matrix; for formation of normal 
equations, 72, 165; notation, 166; 
preparation of matrix, numerical 
examples, 79, 222, 224; reciprocal 
matrix, 19, 91, 159, 160, 162, 217; 
see dso Reciprocal matrix 
Maximum error, 161 
Tobias Matub, 137 
Mean, 18; rapid method of calcula- 
tion, 161; standard error of mean, 
2l, 40; weighted, 19, 24, 26 
Mean square error; of a difference, 
42; of a sum, 42; percentage mean 
square error, 43; propagation of, 39 
Method; of averages, 187; Bruyfae, 
123, 124; Cauchy, 138; least 
squares, 16; selected points, 138; 
Stephan, 121, 123; zero sum, 137 
MiOHBiiB, 128, 212 

Near indeterminacy, 160, 161, 163 
V, A. NnKBABOFS', 169 
Normal deviates in units of standard 
error, 262 

Normal equations; direct solution, 
18; exhibit in symbols, 158; ex- 
ponential, 192 ff- ; for oonditions 
without parameters, 73, 108; for 
curve fitting, 136; for fitting an 
isotherm, 214; for geometric con- 
ditions, 69; for the line, 174 ft.; 
formation, 70 ff., 79, 165; freezing, 
180; general, 66; hyperbola, 206, 
206; in a numerical e-xample, 82; 
number of equations, 66; parabola,. 
188 ff.; tabiiar solution, 19, 33, 66, 
148, ITO; unequivocal solution, 166 
K. A. Nobton, 198 
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Obsenratioiis, rejeotion of, 171 
Order, importance of, 3, 4 
Orthogon^ functions, 10, 172 

A. DH Fokebt Palmhb, 27, 160 
Parabola, 187 

Parameters; adjustment of, 130; 
approximate values, 52, 131, 137, 
153; methods of obtaining, 137, 
138; weight, 167 
Eaui. Peabson, 27, 184, 183 
Percentile, 211 
P. PlZZETH, 142 
Plane triangle, 7, 60, 64 
Positive definite (normal equations), 
68 

Powers of 10, manipulation of, 224 
Precision, 23 
Prediction, 3, 129 
Principle of least squares, 2, 14, 16 
Prior value of <r compared with <r 
(fixl), 230 

Probable error, 169 
Probits, 210 

Product rananoe (covariance), 19, 
160 

Propagation of error; in functions 
of one variable, 37; in functions of 
several variables, 38 
Propagation of mean square error, 
40; of variance, 40; of weight, 40 
UtJTH R. PUPFEU, 22 

Quadratio form, 63 
Quasi center, 181 

Randomness; importance of order, 
3, 4; Shewhort criterion, 0, 220; 
tables for maldng random obser' 
vations, 252 

Reciprocal matrix, 10, 21, 156; 
aa a multiplier, 01, 218; colculsr 
tlon of, 92, 162, 201; for the line, 
176, 185; for the parabola, 100; in a 


numerical example, 92, 217, 224; 
order of, 95; ace aiso Matrix 
Reciprocal solution, 100, 166; in a 
numerical example, 226 
Reduced conditions, 52 
Rejection of observations, 171 
Residual, 14; calculation of, 56, 161, 
158, 216, 230; definition of, 17, 
50; standardized, 22, 23; variance 
of, 127; see also Sum of squares 
Rim total (marginal total), 98 

G. RoniNsoN, 137, 159, 167 

Root mean square error, 42; see also 
Standard eiTor 

S', see Sum of squares 
Fkank S. SALisDunY, 159 
Sample frequencies; adjusted by 
Bruyire method, 124; adjusted by 
least squares, 101 II.; adjusted by 
Stephan method, 121 ; adjustment 
to expected marginal totals, 96 if. 
Sample surveys, 10, 31; of canned 
goods, 240 

Sampling errors, 10, 127, 249 
Sampling ratio, 08 
Sampling variance, 100 
Max Sasuly, 173 
Henky SenuLTz, 100, 170, 177 
Fuancis X. ScHUMACHEn, 231, 238 
Bchwarz-Christoffel inequality, 162 
Second adjustment of parameters, 
52, 180 

Segmentsof a line, adjustment of, 9, 80 
Selected points, method of, 52, 131, 
138 

Sbioeii, 150 

John D. Shea, 173, 176 
Wambb a. Shewhaht, 6, 109, 171, 
220; control chart, 220; criterion 
of randomness, 6, 220 
Significance; scientific, 12; statisti- 
cal, 12, 30; tests of, 30, 169 

H. SiLVEHBTONB (and Aitken), 10 
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Slice totals, 110 

Small errors; in funotioiis of one 
variable, 37 ; in functionB of several 
variables, 38; numerical example, 41 
John H. Smith, 201 
T. Smite, 169, 160 
Solution TTitliout Lagrange mulU- 
pUers, 62 

Stability (randomness), 6 
Standard deviation, 18; rapid method 
of computing, 161 

Standard error, 12, 16, 21; of a 
curve, 167; of a function of the 
parameters, 167; of a mean, 40; 
of adjusted parameters, 167; of 
calculated ordinates, 229 
Standardize re^duals, 22, 23 
Statistical significance, 12, 30 
Fredhsioe F. Sththah, 100, 121, 
123 

R. Mbmbtim Stevtabt, 140 
J. Stbvhns Stock, 32 
Student, 34, 142; Student’s distri- 
bution, 169 

Sum check (comer check), 72, 166 
Sum of squares (S), 14, 20; effect of 
changes in units, 23; formulas for 
S, 68, 163, 164; removed by 
regression, 176; short expression, 
66; special formulas, 68, 163, 164; 
systematic computation, 166 
Surveying problem, 74 ff. 

Systematic solution of normal equa- 
tions, 166; form for computation, 
168 

t test, 34, 169 

Tabular solution of normal equa- 
tions, 19, 33, 66, 148, 160; exhibit 
in symbols, 168 
Taylor’s series, 38, 41, 62, 139 
Tests of significance, 30, 169 
Tippett’s 
262, 266 


Triangle problem, 60; without La- 
grange multipliers, 62 
Tme points, 133 
Tme value, definition of, 49 
1. B. Tuckebman, 161 

Horace S. Uhler, 140 
Unbiased estimate of v, 27, 168 
Unequivocal solution of normal equa- 
tions, 166 

Vacancy; rate, 32; sample survey of, 
31 

Variance; analysis of, 29; of resid- 
uals, 127; propagation of, 40; 
sampling, 100 

Variance coefficient, 19, 21, 22, 46; 
of a function of unit weight, 21; 
of parameters, 160 
Variate, random, 21 

Abraham Wald, 137 
Weight; definition of, 21; for loga- 
rithmic transformatioD, 46, 200; 
of adjusted parameters, 167; of 
function of adjusted angles, 64; 
propagation of, 40 
Weights of functions after adjust- 
ment, 68; numerical examples, 
86, 94; short method of com- 
puting, 90 
L. D. Weld, 86 

E. T. Whittaker (and G, Robin- 
bon), 137, 169, 167 
Benjamin Williamson, 63 
E. B. Wilson, 22 
H. Wouters, 212 

T. W. Wright (and Hatvohd), 66 

Frank Yates, 172 
Theodore Yntema, 201, 202 


numbers, ,.107,-ai8|«98^V z (Fifihfer); 28, 30 
National iiZ^6«,aSth^ 



